Catatan Seekor: Open AI

Overview

OpenAI adalah perusahaan riset AI yang mengembangkan model bahasa canggih seperti GPT (Generative Pre-trained Transformer) dan menyediakan API untuk berbagai aplikasi AI. OpenAI telah merevolusi bidang AI dengan model-model yang dapat memahami dan menghasilkan teks, gambar, dan audio dengan kualitas manusia.

Sejarah dan Perkembangan

• 2015: OpenAI didirikan oleh Elon Musk, Sam Altman, dan lainnya

• 2018: GPT-1 dirilis dengan 117 juta parameter

• 2019: GPT-2 dengan 1.5 miliar parameter

• 2020: GPT-3 dengan 175 miliar parameter

• 2022: ChatGPT dan GPT-3.5-turbo dirilis

• 2023: GPT-4 dengan kemampuan multimodal

• 2024: GPT-4 Turbo dan GPT-4o dengan peningkatan signifikan

Fundamental

OpenAI Models

• GPT-4o: Model bahasa terbaru dan paling canggih dengan kemampuan multimodal

• GPT-4 Turbo: Model yang seimbang antara performa dan biaya

• GPT-3.5-turbo: Model yang cepat dan efisien untuk aplikasi umum

• DALL-E 3: Model untuk generasi gambar dari teks dengan kualitas tinggi

• Whisper: Model untuk speech-to-text dengan akurasi tinggi

• Embeddings: Model untuk representasi vektor teks (text-embedding-ada-002)

• Codex: Model khusus untuk generasi kode (tidak tersedia untuk publik)

API Authentication

import openai

# Set API key
openai.api_key = "your-api-key-here"

# Or use environment variable
import os
openai.api_key = os.getenv("OPENAI_API_KEY")

# For OpenAI Python client v1.0+
from openai import OpenAI
client = OpenAI(api_key="your-api-key-here")

API Rate Limits dan Pricing

• GPT-4o: $5.00 per 1M input tokens, $15.00 per 1M output tokens

• GPT-4 Turbo: $10.00 per 1M input tokens, $30.00 per 1M output tokens

• GPT-3.5-turbo: $0.50 per 1M input tokens, $1.50 per 1M output tokens

• DALL-E 3: $0.040 per image (1024x1024), $0.080 per image (1024x1792)

• Whisper: $0.006 per minute

Text Generation

Basic Chat Completion

import openai

def chat_with_gpt(prompt, model="gpt-3.5-turbo"):
    response = openai.ChatCompletion.create(
        model=model,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": prompt}
        ],
        max_tokens=1000,
        temperature=0.7
    )
    return response.choices[0].message.content

# Usage
response = chat_with_gpt("Explain quantum computing in simple terms")
print(response)

Advanced Chat with Context

def multi_turn_chat(conversation_history, new_message):
    # Add new message to history
    conversation_history.append({"role": "user", "content": new_message})
    
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=conversation_history,
        max_tokens=500,
        temperature=0.8
    )
    
    # Add assistant response to history
    conversation_history.append({
        "role": "assistant", 
        "content": response.choices[0].message.content
    })
    
    return response.choices[0].message.content

# Initialize conversation
conversation = [
    {"role": "system", "content": "You are a coding tutor. Help users learn programming."}
]

# Multi-turn conversation
response1 = multi_turn_chat(conversation, "What is Python?")
response2 = multi_turn_chat(conversation, "How do I create a function?")

Text Completion (Legacy)

def text_completion(prompt, model="text-davinci-003"):
    response = openai.Completion.create(
        model=model,
        prompt=prompt,
        max_tokens=150,
        temperature=0.8,
        top_p=1,
        frequency_penalty=0,
        presence_penalty=0
    )
    return response.choices[0].text.strip()

# Usage
prompt = "Write a short story about a robot learning to paint:"
story = text_completion(prompt)
print(story)

Image Generation

DALL-E 3 Image Generation

def generate_image(prompt, size="1024x1024", quality="standard"):
    response = openai.Image.create(
        prompt=prompt,
        n=1,
        size=size,
        quality=quality,
        style="vivid"  # or "natural"
    )
    return response.data[0].url

# Usage
image_url = generate_image("A serene mountain landscape at sunset with a lake in the foreground")
print(f"Generated image: {image_url}")

Image Variations

def create_image_variation(image_path, n=1, size="1024x1024"):
    with open(image_path, "rb") as image_file:
        response = openai.Image.create_variation(
            image=image_file,
            n=n,
            size=size
        )
    return [img.url for img in response.data]

# Usage
variations = create_image_variation("path/to/your/image.png")
for i, url in enumerate(variations):
    print(f"Variation {i+1}: {url}")

Image Editing

def edit_image(image_path, mask_path, prompt, n=1, size="1024x1024"):
    with open(image_path, "rb") as image_file, open(mask_path, "rb") as mask_file:
        response = openai.Image.create_edit(
            image=image_file,
            mask=mask_file,
            prompt=prompt,
            n=n,
            size=size
        )
    return [img.url for img in response.data]

Speech-to-Text dengan Whisper

Basic Transcription

def transcribe_audio(audio_file_path):
    with open(audio_file_path, "rb") as audio_file:
        transcript = openai.Audio.transcribe("whisper-1", audio_file)
    return transcript.text

# Usage
transcript = transcribe_audio("path/to/audio.mp3")
print(f"Transcription: {transcript}")

Advanced Transcription

def transcribe_with_options(audio_file_path, language="en", prompt="This is a technical discussion about AI"):
    with open(audio_file_path, "rb") as audio_file:
        transcript = openai.Audio.transcribe(
            "whisper-1",
            audio_file,
            language=language,
            prompt=prompt,
            response_format="verbose_json"
        )
    return transcript

# Usage
result = transcribe_with_options("audio.mp3", language="id")
print(f"Text: {result.text}")
print(f"Language: {result.language}")
print(f"Duration: {result.duration}")

Text Embeddings

Creating Embeddings

def get_embeddings(text, model="text-embedding-ada-002"):
    response = openai.Embedding.create(
        input=text,
        model=model
    )
    return response.data[0].embedding

# Usage
text = "This is a sample text for embedding"
embedding = get_embeddings(text)
print(f"Embedding dimension: {len(embedding)}")

Semantic Search

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity

def semantic_search(query, documents, model="text-embedding-ada-002"):
    # Get query embedding
    query_embedding = get_embeddings(query, model)
    
    # Get document embeddings
    document_embeddings = [get_embeddings(doc, model) for doc in documents]
    
    # Calculate similarities
    similarities = [cosine_similarity([query_embedding], [doc_emb])[0][0] 
                   for doc_emb in document_embeddings]
    
    # Return sorted results
    results = list(zip(documents, similarities))
    results.sort(key=lambda x: x[1], reverse=True)
    return results

# Usage
documents = [
    "Machine learning is a subset of artificial intelligence",
    "Python is a popular programming language",
    "Deep learning uses neural networks with multiple layers"
]
query = "What is artificial intelligence?"
results = semantic_search(query, documents)
for doc, score in results:
    print(f"Score: {score:.3f} - {doc}")

Fine-tuning

Preparing Data for Fine-tuning

import json

def prepare_fine_tuning_data(conversations, output_file="fine_tune_data.jsonl"):
    """Prepare conversations for fine-tuning"""
    with open(output_file, 'w') as f:
        for conversation in conversations:
            # Convert to OpenAI format
            formatted_data = {
                "messages": conversation
            }
            f.write(json.dumps(formatted_data) + '\n')
    
    print(f"Data prepared and saved to {output_file}")

# Example conversations
conversations = [
    [
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "How do I create a function in Python?"},
        {"role": "assistant", "content": "To create a function in Python, use the 'def' keyword..."}
    ],
    [
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "What is a loop?"},
        {"role": "assistant", "content": "A loop is a programming construct that repeats a block of code..."}
    ]
]

prepare_fine_tuning_data(conversations)

Creating Fine-tuned Model

def create_fine_tuned_model(training_file_id, validation_file_id=None, model="gpt-3.5-turbo"):
    """Create a fine-tuned model"""
    response = openai.FineTuningJob.create(
        training_file=training_file_id,
        validation_file=validation_file_id,
        model=model,
        hyperparameters={
            "n_epochs": 3,
            "batch_size": 3,
            "learning_rate_multiplier": 0.1
        }
    )
    return response.id

# Usage
job_id = create_fine_tuned_model("file-abc123")
print(f"Fine-tuning job created: {job_id}")

Best Practices

1. Prompt Engineering

def create_effective_prompt(task, context, examples=None):
    """Create an effective prompt for better results"""
    prompt = f"""
Task: {task}

Context: {context}

Instructions:
- Be concise and accurate
- Provide practical examples when relevant
- Use clear, simple language

"""
    
    if examples:
        prompt += f"Examples:\n{examples}\n\n"
    
    prompt += "Response:"
    return prompt

# Usage
task = "Explain machine learning"
context = "For a beginner programmer"
examples = "1. Supervised learning: training with labeled data\n2. Unsupervised learning: finding patterns in data"
prompt = create_effective_prompt(task, context, examples)

2. Error Handling

import time
from openai.error import RateLimitError, APIError, InvalidRequestError

def robust_api_call(func, max_retries=3, delay=1):
    """Robust API call with retry logic"""
    for attempt in range(max_retries):
        try:
            return func()
        except RateLimitError:
            if attempt < max_retries - 1:
                time.sleep(delay * (2 ** attempt))  # Exponential backoff
                continue
            raise
        except (APIError, InvalidRequestError) as e:
            print(f"API Error: {e}")
            raise
        except Exception as e:
            print(f"Unexpected error: {e}")
            raise

# Usage
def api_function():
    return openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Hello"}]
    )

result = robust_api_call(api_function)

3. Cost Optimization

def estimate_tokens(text):
    """Rough estimation of token count"""
    return len(text.split()) * 1.3  # Rough approximation

def optimize_prompt(prompt, max_tokens=1000):
    """Optimize prompt to reduce token usage"""
    if estimate_tokens(prompt) > max_tokens:
        # Truncate or summarize prompt
        words = prompt.split()
        return " ".join(words[:int(max_tokens/1.3)])
    return prompt

# Usage
long_prompt = "Very long prompt..." * 100
optimized = optimize_prompt(long_prompt, max_tokens=500)
print(f"Original tokens: ~{estimate_tokens(long_prompt):.0f}")
print(f"Optimized tokens: ~{estimate_tokens(optimized):.0f}")

Integrasi dengan Framework

Flask Web Application

from flask import Flask, request, jsonify
import openai

app = Flask(__name__)
openai.api_key = "your-api-key"

@app.route('/chat', methods=['POST'])
def chat():
    try:
        data = request.json
        user_message = data.get('message', '')
        
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=[
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": user_message}
            ],
            max_tokens=500,
            temperature=0.7
        )
        
        return jsonify({
            'response': response.choices[0].message.content,
            'status': 'success'
        })
    
    except Exception as e:
        return jsonify({
            'error': str(e),
            'status': 'error'
        }), 500

if __name__ == '__main__':
    app.run(debug=True)

Streamlit Application

import streamlit as st
import openai

st.title("OpenAI Chat Assistant")

# Sidebar for configuration
with st.sidebar:
    openai.api_key = st.text_input("OpenAI API Key", type="password")
    model = st.selectbox("Model", ["gpt-3.5-turbo", "gpt-4"])
    temperature = st.slider("Temperature", 0.0, 1.0, 0.7)

# Main chat interface
if "messages" not in st.session_state:
    st.session_state.messages = []

for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

if prompt := st.chat_input("What would you like to know?"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        if openai.api_key:
            try:
                response = openai.ChatCompletion.create(
                    model=model,
                    messages=st.session_state.messages,
                    temperature=temperature
                )
                assistant_response = response.choices[0].message.content
                st.markdown(assistant_response)
                st.session_state.messages.append({"role": "assistant", "content": assistant_response})
            except Exception as e:
                st.error(f"Error: {e}")
        else:
            st.error("Please enter your OpenAI API key in the sidebar.")

Monitoring dan Analytics

Usage Tracking

import time
from datetime import datetime, timedelta

class OpenAIMonitor:
    def __init__(self):
        self.usage_log = []
        self.cost_log = []
    
    def log_request(self, model, input_tokens, output_tokens, cost):
        """Log API request details"""
        log_entry = {
            'timestamp': datetime.now(),
            'model': model,
            'input_tokens': input_tokens,
            'output_tokens': output_tokens,
            'cost': cost
        }
        self.usage_log.append(log_entry)
    
    def get_daily_usage(self, date=None):
        """Get usage statistics for a specific date"""
        if date is None:
            date = datetime.now().date()
        
        daily_logs = [log for log in self.usage_log 
                     if log['timestamp'].date() == date]
        
        total_cost = sum(log['cost'] for log in daily_logs)
        total_requests = len(daily_logs)
        
        return {
            'date': date,
            'total_cost': total_cost,
            'total_requests': total_requests,
            'requests': daily_logs
        }
    
    def get_cost_breakdown(self, days=7):
        """Get cost breakdown for the last N days"""
        end_date = datetime.now().date()
        start_date = end_date - timedelta(days=days)
        
        breakdown = {}
        current_date = start_date
        
        while current_date <= end_date:
            daily_usage = self.get_daily_usage(current_date)
            breakdown[current_date.isoformat()] = daily_usage['total_cost']
            current_date += timedelta(days=1)
        
        return breakdown

# Usage
monitor = OpenAIMonitor()

# Log a request
monitor.log_request(
    model="gpt-3.5-turbo",
    input_tokens=100,
    output_tokens=50,
    cost=0.0001
)

# Get daily usage
today_usage = monitor.get_daily_usage()
print(f"Today's cost: ${today_usage['total_cost']:.4f}")

# Get weekly breakdown
weekly_costs = monitor.get_cost_breakdown(days=7)
for date, cost in weekly_costs.items():
    print(f"{date}: ${cost:.4f}")

Referensi

Dokumentasi Resmi

• OpenAI API Documentation - Dokumentasi lengkap API

• OpenAI Python Library - Library Python resmi

• OpenAI Cookbook - Contoh dan resep penggunaan

• OpenAI Pricing - Informasi harga lengkap

Tutorial dan Artikel

• OpenAI API Tutorial - Tutorial cepat memulai

• Best Practices for Prompt Engineering - Best practices prompt engineering

• Fine-tuning Guide - Panduan fine-tuning

• Safety Best Practices - Praktik keamanan

Komunitas dan Forum

• OpenAI Community Forum - Forum komunitas resmi

• OpenAI Discord - Discord server OpenAI

• Stack Overflow OpenAI Tag - Q&A Stack Overflow

• Reddit r/OpenAI - Subreddit OpenAI

Tools dan Framework

• LangChain - Framework untuk aplikasi LLM

• OpenAI Whisper - Implementasi Whisper

• DALL-E 2 - Implementasi DALL-E

• OpenAI Gym - Toolkit untuk reinforcement learning

Buku dan Kursus

• "Prompt Engineering for Developers" - OpenAI Course

• "Building LLM Applications" - LangChain Course

• "AI Engineering" - DeepLearning.AI Course

• "OpenAI API Mastery" - Udemy Course

Tips dan Trik

1. Optimasi Prompt

• Gunakan instruksi yang jelas dan spesifik

• Berikan contoh yang relevan

• Gunakan format yang konsisten

• Test prompt dengan berbagai input

2. Manajemen Biaya

• Monitor penggunaan API secara regular

• Gunakan model yang sesuai dengan kebutuhan

• Implementasikan caching untuk response yang sering digunakan

• Optimasi prompt untuk mengurangi token usage

3. Keamanan dan Privasi

• Jangan kirim data sensitif ke API

• Implementasikan rate limiting

• Monitor dan log semua request

• Gunakan environment variables untuk API keys

4. Performa

• Gunakan async/await untuk multiple requests

• Implementasikan connection pooling

• Cache embeddings dan response yang sering digunakan

• Monitor latency dan throughput

OpenAI telah membuka era baru dalam pengembangan AI dengan API yang powerful dan mudah digunakan. Dengan memahami fundamental dan best practices, Anda dapat membangun aplikasi AI yang inovatif dan efektif.