# Advanced Topics

## 📚 Overview

Halaman ini berisi topik-topik lanjutan dalam Machine Learning yang mencakup workflow automation, MLOps, advanced architectures, dan teknik-teknik modern yang digunakan dalam production ML systems.

## 🔄 Workflow Automation & N8N

### **N8N for ML Workflows**

N8N sebagai platform workflow automation untuk ML:

#### **1. N8N Fundamentals**

* **Platform**: [N8N](https://n8n.io/)
* **Type**: Workflow automation platform
* **Features**:
  * Visual workflow builder
  * 500+ integrations
  * Self-hosted option
  * JavaScript/TypeScript support
* **Best For**: ML pipeline automation, data workflows
* **Pros**: Visual interface, extensive integrations, self-hosted
* **Cons**: Learning curve, requires workflow design skills
* **Use Cases**: Data pipeline automation, ML workflow orchestration
* **Rating**: ⭐⭐⭐⭐⭐ (5/5)

#### **2. ML Workflow Examples with N8N**

Contoh workflow ML menggunakan N8N:

**Data Collection Workflow**

```javascript
// N8N workflow for automated data collection
{
  "nodes": [
    {
      "type": "n8n-nodes-base.httpRequest",
      "position": [100, 100],
      "parameters": {
        "url": "https://api.example.com/data",
        "method": "GET"
      }
    },
    {
      "type": "n8n-nodes-base.dataTransform",
      "position": [300, 100],
      "parameters": {
        "transform": "cleanData"
      }
    },
    {
      "type": "n8n-nodes-base.postgres",
      "position": [500, 100],
      "parameters": {
        "operation": "insert",
        "table": "raw_data"
      }
    }
  ]
}
```

**Model Training Workflow**

```javascript
// N8N workflow for automated model training
{
  "nodes": [
    {
      "type": "n8n-nodes-base.postgres",
      "position": [100, 100],
      "parameters": {
        "operation": "select",
        "table": "training_data"
      }
    },
    {
      "type": "n8n-nodes-base.httpRequest",
      "position": [300, 100],
      "parameters": {
        "url": "https://ml-training-service.com/train",
        "method": "POST",
        "body": "{{ $json }}"
      }
    },
    {
      "type": "n8n-nodes-base.slack",
      "position": [500, 100],
      "parameters": {
        "channel": "#ml-alerts",
        "text": "Model training completed: {{ $json.model_id }}"
      }
    }
  ]
}
```

#### **3. N8N ML Integrations**

Integrasi N8N dengan tools ML:

**Hugging Face Integration**

* **Model Inference**: Automate model predictions
* **Model Training**: Trigger training jobs
* **Dataset Management**: Automate data pipeline
* **API Management**: Handle model serving

**MLflow Integration**

* **Experiment Tracking**: Log ML experiments
* **Model Registry**: Manage model versions
* **Deployment**: Automate model deployment
* **Monitoring**: Track model performance

**Kubeflow Integration**

* **Pipeline Orchestration**: Manage ML pipelines
* **Resource Management**: Automate resource allocation
* **Training Jobs**: Schedule and monitor training
* **Model Serving**: Deploy models automatically

### **Advanced Workflow Patterns**

Pattern workflow lanjutan untuk ML:

#### **4. Event-Driven ML Workflows**

Workflow berbasis event untuk ML:

```javascript
// Event-driven workflow for real-time ML
{
  "trigger": "webhook",
  "nodes": [
    {
      "type": "n8n-nodes-base.webhook",
      "parameters": {
        "path": "ml-prediction",
        "httpMethod": "POST"
      }
    },
    {
      "type": "n8n-nodes-base.if",
      "parameters": {
        "conditions": {
          "string": [
            {
              "value1": "{{ $json.data_type }}",
              "operation": "equals",
              "value2": "image"
            }
          ]
        }
      }
    },
    {
      "type": "n8n-nodes-base.httpRequest",
      "parameters": {
        "url": "https://vision-model.com/predict",
        "method": "POST"
      }
    }
  ]
}
```

#### **5. Batch Processing Workflows**

Workflow untuk batch processing:

```javascript
// Batch processing workflow for large datasets
{
  "nodes": [
    {
      "type": "n8n-nodes-base.cron",
      "parameters": {
        "rule": "0 2 * * *" // Daily at 2 AM
      }
    },
    {
      "type": "n8n-nodes-base.postgres",
      "parameters": {
        "operation": "select",
        "query": "SELECT * FROM raw_data WHERE processed = false"
      }
    },
    {
      "type": "n8n-nodes-base.splitInBatches",
      "parameters": {
        "batchSize": 1000
      }
    },
    {
      "type": "n8n-nodes-base.httpRequest",
      "parameters": {
        "url": "https://batch-processor.com/process",
        "method": "POST"
      }
    }
  ]
}
```

## 🤖 Advanced ML Architectures

### **Transformer Architecture Deep Dive**

Implementasi detail Transformer:

#### **6. Multi-Head Attention Implementation**

```python
import torch
import torch.nn as nn
import math

class MultiHeadAttention(nn.Module):
    def __init__(self, d_model, num_heads, dropout=0.1):
        super().__init__()
        assert d_model % num_heads == 0
        
        self.d_model = d_model
        self.num_heads = num_heads
        self.d_k = d_model // num_heads
        
        self.w_q = nn.Linear(d_model, d_model)
        self.w_k = nn.Linear(d_model, d_model)
        self.w_v = nn.Linear(d_model, d_model)
        self.w_o = nn.Linear(d_model, d_model)
        
        self.dropout = nn.Dropout(dropout)
        
    def scaled_dot_product_attention(self, Q, K, V, mask=None):
        # Calculate attention scores
        scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(self.d_k)
        
        if mask is not None:
            scores = scores.masked_fill(mask == 0, -1e9)
        
        # Apply softmax
        attention_weights = torch.softmax(scores, dim=-1)
        attention_weights = self.dropout(attention_weights)
        
        # Apply attention weights to values
        output = torch.matmul(attention_weights, V)
        return output, attention_weights
    
    def forward(self, query, key, value, mask=None):
        batch_size = query.size(0)
        
        # Linear transformations and reshape
        Q = self.w_q(query).view(batch_size, -1, self.num_heads, self.d_k).transpose(1, 2)
        K = self.w_k(key).view(batch_size, -1, self.num_heads, self.d_k).transpose(1, 2)
        V = self.w_v(value).view(batch_size, -1, self.num_heads, self.d_k).transpose(1, 2)
        
        # Apply attention
        attention_output, attention_weights = self.scaled_dot_product_attention(Q, K, V, mask)
        
        # Reshape and apply final linear transformation
        attention_output = attention_output.transpose(1, 2).contiguous().view(
            batch_size, -1, self.d_model
        )
        output = self.w_o(attention_output)
        
        return output, attention_weights
```

#### **7. Transformer Block Implementation**

```python
class TransformerBlock(nn.Module):
    def __init__(self, d_model, num_heads, d_ff, dropout=0.1):
        super().__init__()
        
        self.attention = MultiHeadAttention(d_model, num_heads, dropout)
        self.norm1 = nn.LayerNorm(d_model)
        self.norm2 = nn.LayerNorm(d_model)
        
        self.feed_forward = nn.Sequential(
            nn.Linear(d_model, d_ff),
            nn.ReLU(),
            nn.Dropout(dropout),
            nn.Linear(d_ff, d_model)
        )
        
        self.dropout = nn.Dropout(dropout)
        
    def forward(self, x, mask=None):
        # Multi-head attention with residual connection
        attn_output, _ = self.attention(x, x, x, mask)
        x = self.norm1(x + self.dropout(attn_output))
        
        # Feed-forward network with residual connection
        ff_output = self.feed_forward(x)
        x = self.norm2(x + self.dropout(ff_output))
        
        return x
```

### **Advanced Neural Network Architectures**

Arsitektur neural network lanjutan:

#### **8. ResNet Implementation**

```python
class ResidualBlock(nn.Module):
    def __init__(self, in_channels, out_channels, stride=1):
        super().__init__()
        
        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, 
                               stride=stride, padding=1, bias=False)
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3,
                               stride=1, padding=1, bias=False)
        self.bn2 = nn.BatchNorm2d(out_channels)
        
        self.shortcut = nn.Sequential()
        if stride != 1 or in_channels != out_channels:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_channels, out_channels, kernel_size=1, 
                          stride=stride, bias=False),
                nn.BatchNorm2d(out_channels)
            )
    
    def forward(self, x):
        residual = x
        
        out = torch.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        
        out += self.shortcut(residual)
        out = torch.relu(out)
        
        return out

class ResNet(nn.Module):
    def __init__(self, block, num_blocks, num_classes=1000):
        super().__init__()
        self.in_channels = 64
        
        self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
        self.bn1 = nn.BatchNorm2d(64)
        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
        
        self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)
        self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)
        self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)
        self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)
        
        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.fc = nn.Linear(512, num_classes)
    
    def _make_layer(self, block, out_channels, num_blocks, stride):
        strides = [stride] + [1] * (num_blocks - 1)
        layers = []
        for stride in strides:
            layers.append(block(self.in_channels, out_channels, stride))
            self.in_channels = out_channels
        return nn.Sequential(*layers)
    
    def forward(self, x):
        x = torch.relu(self.bn1(self.conv1(x)))
        x = self.maxpool(x)
        
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)
        
        x = self.avgpool(x)
        x = x.view(x.size(0), -1)
        x = self.fc(x)
        
        return x
```

## 🔧 Advanced MLOps & Production

### **Model Serving & Deployment**

Advanced deployment strategies:

#### **9. Model Serving with FastAPI**

```python
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel
import uvicorn

app = FastAPI(title="ML Model API", version="1.0.0")

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")
model.eval()

class PredictionRequest(BaseModel):
    text: str
    max_length: int = 512

class PredictionResponse(BaseModel):
    prediction: list
    confidence: float
    processing_time: float

@app.post("/predict", response_model=PredictionResponse)
async def predict(request: PredictionRequest):
    try:
        import time
        start_time = time.time()
        
        # Tokenize input
        inputs = tokenizer(
            request.text,
            return_tensors="pt",
            max_length=request.max_length,
            truncation=True,
            padding=True
        )
        
        # Get predictions
        with torch.no_grad():
            outputs = model(**inputs)
            embeddings = outputs.last_hidden_state.mean(dim=1)
            predictions = F.softmax(embeddings, dim=-1)
        
        processing_time = time.time() - start_time
        
        return PredictionResponse(
            prediction=predictions.tolist(),
            confidence=float(predictions.max()),
            processing_time=processing_time
        )
    
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

@app.get("/health")
async def health_check():
    return {"status": "healthy", "model_loaded": model is not None}

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)
```

#### **10. Kubernetes Deployment for ML Models**

```yaml
# kubernetes-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ml-model-deployment
  labels:
    app: ml-model
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ml-model
  template:
    metadata:
      labels:
        app: ml-model
    spec:
      containers:
      - name: ml-model
        image: your-registry/ml-model:latest
        ports:
        - containerPort: 8000
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5
        env:
        - name: MODEL_PATH
          value: "/models/bert-model"
        - name: LOG_LEVEL
          value: "INFO"
        volumeMounts:
        - name: model-storage
          mountPath: /models
      volumes:
      - name: model-storage
        persistentVolumeClaim:
          claimName: ml-model-pvc

---
apiVersion: v1
kind: Service
metadata:
  name: ml-model-service
spec:
  selector:
    app: ml-model
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8000
  type: LoadBalancer
```

### **Advanced Monitoring & Observability**

Monitoring lanjutan untuk ML systems:

#### **11. ML Model Monitoring with Prometheus**

```python
from prometheus_client import Counter, Histogram, Gauge
import time
import functools

# Metrics
PREDICTION_COUNTER = Counter('ml_predictions_total', 'Total predictions made')
PREDICTION_DURATION = Histogram('ml_prediction_duration_seconds', 'Prediction duration')
MODEL_ACCURACY = Gauge('ml_model_accuracy', 'Model accuracy')
MODEL_MEMORY_USAGE = Gauge('ml_model_memory_bytes', 'Model memory usage')

def monitor_prediction(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start_time = time.time()
        
        try:
            result = func(*args, **kwargs)
            PREDICTION_COUNTER.inc()
            return result
        except Exception as e:
            # Log error metrics
            raise e
        finally:
            duration = time.time() - start_time
            PREDICTION_DURATION.observe(duration)
    
    return wrapper

@monitor_prediction
def predict_with_monitoring(text):
    # Your prediction logic here
    time.sleep(0.1)  # Simulate processing
    return {"prediction": "sample_result"}

# Update model metrics
def update_model_metrics(accuracy, memory_usage):
    MODEL_ACCURACY.set(accuracy)
    MODEL_MEMORY_USAGE.set(memory_usage)
```

## 🧠 Advanced ML Techniques

### **Federated Learning**

Implementasi federated learning:

#### **12. Federated Learning Client**

```python
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import copy

class FederatedClient:
    def __init__(self, model, client_id, local_data):
        self.model = copy.deepcopy(model)
        self.client_id = client_id
        self.local_data = local_data
        self.optimizer = torch.optim.SGD(self.model.parameters(), lr=0.01)
        self.criterion = nn.CrossEntropyLoss()
    
    def train_local(self, epochs=5):
        """Train model on local data"""
        self.model.train()
        
        for epoch in range(epochs):
            for batch_idx, (data, target) in enumerate(self.local_data):
                self.optimizer.zero_grad()
                output = self.model(data)
                loss = self.criterion(output, target)
                loss.backward()
                self.optimizer.step()
        
        return copy.deepcopy(self.model.state_dict())
    
    def get_model_update(self, global_model_state):
        """Get model update relative to global model"""
        local_state = self.model.state_dict()
        update = {}
        
        for key in local_state:
            update[key] = local_state[key] - global_model_state[key]
        
        return update

class FederatedServer:
    def __init__(self, global_model):
        self.global_model = global_model
        self.clients = []
    
    def add_client(self, client):
        self.clients.append(client)
    
    def aggregate_models(self, client_updates):
        """Aggregate client model updates using FedAvg"""
        global_state = self.global_model.state_dict()
        
        # Average the updates
        for key in global_state:
            global_state[key] = torch.mean(
                torch.stack([update[key] for update in client_updates]), dim=0
            )
        
        self.global_model.load_state_dict(global_state)
        return self.global_model.state_dict()
    
    def federated_round(self):
        """Execute one federated learning round"""
        client_updates = []
        
        # Collect updates from all clients
        for client in self.clients:
            update = client.get_model_update(self.global_model.state_dict())
            client_updates.append(update)
        
        # Aggregate updates
        new_global_state = self.aggregate_models(client_updates)
        
        # Update clients with new global model
        for client in self.clients:
            client.model.load_state_dict(new_global_state)
        
        return new_global_state
```

### **Advanced Optimization Techniques**

Teknik optimisasi lanjutan:

#### **13. Custom Optimizer Implementation**

```python
import torch
import torch.optim as optim
import math

class AdaBelief(optim.Optimizer):
    """
    AdaBelief Optimizer implementation
    Paper: AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
    """
    
    def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-16,
                 weight_decay=0, amsgrad=False, rectify=True):
        if not 0.0 <= lr:
            raise ValueError(f"Invalid learning rate: {lr}")
        if not 0.0 <= eps:
            raise ValueError(f"Invalid epsilon value: {eps}")
        if not 0.0 <= betas[0] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 0: {betas[0]}")
        if not 0.0 <= betas[1] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 1: {betas[1]}")
        if not 0.0 <= weight_decay:
            raise ValueError(f"Invalid weight_decay value: {weight_decay}")
        
        defaults = dict(lr=lr, betas=betas, eps=eps,
                       weight_decay=weight_decay, amsgrad=amsgrad, rectify=rectify)
        super(AdaBelief, self).__init__(params, defaults)
    
    def __setstate__(self, state):
        super(AdaBelief, self).__setstate__(state)
        for group in self.param_groups:
            group.setdefault('amsgrad', False)
            group.setdefault('rectify', True)
    
    def step(self, closure=None):
        loss = None
        if closure is not None:
            loss = closure()
        
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is None:
                    continue
                
                grad = p.grad.data
                if grad.is_sparse:
                    raise RuntimeError('AdaBelief does not support sparse gradients')
                
                state = self.state[p]
                
                # State initialization
                if len(state) == 0:
                    state['step'] = 0
                    # Exponential moving average of gradient values
                    state['exp_avg'] = torch.zeros_like(p.data)
                    # Exponential moving average of squared gradient values
                    state['exp_avg_sq'] = torch.zeros_like(p.data)
                    if group['amsgrad']:
                        # Maintains max of all exp_avg_sq until now
                        state['max_exp_avg_sq'] = torch.zeros_like(p.data)
                
                exp_avg, exp_avg_sq = state['exp_avg'], state['exp_avg_sq']
                if group['amsgrad']:
                    max_exp_avg_sq = state['max_exp_avg_sq']
                beta1, beta2 = group['betas']
                
                state['step'] += 1
                bias_correction1 = 1 - beta1 ** state['step']
                bias_correction2 = 1 - beta2 ** state['step']
                
                if group['weight_decay'] != 0:
                    grad = grad.add(group['weight_decay'], p.data)
                
                # Decay the first and second moment running average coefficient
                exp_avg.mul_(beta1).add_(1 - beta1, grad)
                exp_avg_sq.mul_(beta2).addcmul_(1 - beta2, grad, grad)
                
                if group['amsgrad']:
                    # Maintains the maximum of all 2nd moment running avg. until now
                    torch.max(max_exp_avg_sq, exp_avg_sq, out=max_exp_avg_sq)
                    # Use the max. for normalizing running avg. of gradient
                    denom = max_exp_avg_sq.sqrt().add_(group['eps'])
                else:
                    denom = exp_avg_sq.sqrt().add_(group['eps'])
                
                step_size = group['lr'] / bias_correction1
                bias_correction2_sqrt = math.sqrt(bias_correction2)
                
                if group['rectify']:
                    # Rectified update
                    rect = math.sqrt((1 - beta2 ** state['step']) / (1 - beta1 ** state['step']))
                    step_size = step_size * rect
                
                p.data.addcdiv_(-step_size, exp_avg, denom)
        
        return loss
```

## 🚀 Getting Started with Advanced Topics

### **For Intermediate ML Practitioners**

1. **Start with**: N8N workflows, basic MLOps
2. **Then**: Advanced architectures, custom implementations
3. **Finally**: Production deployment, monitoring

### **For Advanced ML Engineers**

1. **Start with**: Custom architectures, advanced optimization
2. **Then**: Federated learning, advanced MLOps
3. **Finally**: Research contributions, tool development

## 💡 Best Practices

### **Workflow Design**

1. **Start simple**: Begin with basic workflows
2. **Modular design**: Break complex workflows into components
3. **Error handling**: Implement proper error handling and retries
4. **Monitoring**: Add monitoring and alerting to workflows
5. **Documentation**: Document workflow logic and dependencies

### **Production Deployment**

1. **Containerization**: Use Docker for consistent environments
2. **Orchestration**: Use Kubernetes for scaling and management
3. **Monitoring**: Implement comprehensive monitoring and alerting
4. **Security**: Follow security best practices for ML systems
5. **Testing**: Implement thorough testing for ML pipelines

***

*Last updated: December 2024* *Contributors: \[Your Name]*

**Note**: Advanced topics require strong ML foundation. Always test implementations thoroughly before production use.