Vibe Coding
IDE Integration

# vibex.sh Log Format Rules

When generating logs or debugging output, always format data to be compatible with vibex.sh log monitoring. vibex.sh supports multiple log formats and intelligently parses them to extract metrics, context, and structured data.

## Core Principles

1. **Hybrid JSON Structure**: vibex.sh uses a hybrid JSON structure that combines metrics, logs, and context
2. **Multiple Log Types**: Support for JSON, text, web server logs, stack traces, and more
3. **Auto-Detection**: The system automatically detects and parses various log formats
4. **Structured Data**: Numeric values become metrics, strings become categorical data, nested objects become facets

## Supported Log Types

vibex.sh supports 13 different log types, each optimized for specific use cases:

### 1. JSON Logs (Default)
Structured JSON logs with hybrid structure. Best for application logs, metrics, and structured data.

### 2. Text Logs
Plain text logs that will be parsed by smart pattern detection. Best for unstructured logs.

### 3. Web Server Logs
Nginx, Apache, and other web server access logs. Automatically extracts HTTP method, status, path, query parameters.

### 4. Load Balancer Logs
HAProxy, AWS ALB, and other load balancer logs.

### 5. Stack Traces
Error stack traces with file, line, and function information.

### 6. Firewall Logs
iptables, pfSense, Cisco ASA firewall logs.

### 7. Kubernetes Logs
Kubernetes pod and container logs.

### 8. Docker Logs
Docker container logs.

### 9. Network Logs
tcpdump, wireshark, and other network packet logs.

### 10. Key-Value Logs
Key-value pair formatted logs (e.g., `key1=value1 key2=value2`).

### 11. JSON-in-Text
JSON objects embedded in text logs.

### 12. Smart Pattern
Multi-language pattern matching for various log formats.

### 13. Raw Logs
Fallback parser for any other log format.

## Hybrid JSON Structure

When using JSON logs, vibex.sh expects a hybrid structure that combines multiple data types:

```json
{
  "message": "Human readable log message",
  "level": "info|warn|error|debug",
  "metrics": {
    "cpu": 45,
    "memory": 78,
    "latency_ms": 200,
    "requests_per_sec": 1200
  },
  "context": {
    "trace_id": "abc-123-def",
    "user_id": "u_999",
    "request_id": "req_456",
    "environment": "production"
  },
  "timestamp": "2024-01-15T10:00:00Z"
}
```

### Field Guidelines

- **message**: Human-readable log message (optional but recommended)
- **level**: Log level - `info`, `warn`, `error`, `debug` (defaults to `debug` if not set)
- **metrics**: Numeric values that will be charted (cpu, memory, latency, etc.)
- **context**: Indexed fields for filtering (trace_id, user_id, request_id, etc.)
- **timestamp**: ISO 8601 format (auto-filled if missing)

## Detailed Examples by Log Type

### JSON Logs - Application Metrics

```json
{"timestamp": "2024-01-15T10:00:00Z", "level": "info", "message": "User logged in", "user_id": 123, "cpu": 45, "memory": 78}
{"timestamp": "2024-01-15T10:00:01Z", "level": "error", "error_code": "DB_001", "message": "Database connection failed", "retry_count": 3}
{"timestamp": "2024-01-15T10:00:02Z", "cpu": 45, "memory": 78, "requests": 1200, "latency_ms": 200}
{"timestamp": "2024-01-15T10:00:03Z", "level": "info", "message": "API request completed", "duration_ms": 150, "status_code": 200, "path": "/api/users", "method": "GET"}
```

### JSON Logs - With Context for Tracing

```json
{"timestamp": "2024-01-15T10:00:00Z", "level": "info", "message": "Processing payment", "trace_id": "trace-abc-123", "user_id": "u_999", "order_id": "order_456", "amount": 99.99}
{"timestamp": "2024-01-15T10:00:01Z", "level": "info", "message": "Payment gateway called", "trace_id": "trace-abc-123", "duration_ms": 250, "gateway": "stripe"}
{"timestamp": "2024-01-15T10:00:02Z", "level": "info", "message": "Payment confirmed", "trace_id": "trace-abc-123", "user_id": "u_999", "transaction_id": "txn_789"}
```

### JSON Logs - Performance Metrics

```json
{"timestamp": "2024-01-15T10:00:00Z", "cpu": 45.2, "memory": 78.5, "disk_io": 1200, "network_io": 3400}
{"timestamp": "2024-01-15T10:00:01Z", "level": "info", "message": "Cache hit", "cache_hit_rate": 0.95, "cache_size_mb": 512}
{"timestamp": "2024-01-15T10:00:02Z", "level": "info", "message": "Database query", "query_duration_ms": 45, "rows_returned": 100, "table": "users"}
```

### Text Logs - Plain Text

```
Application started successfully
User [email protected] logged in from IP 192.168.1.100
High memory usage detected: 85%
Database connection pool exhausted
```

### Web Server Logs - Nginx/Apache Format

```
127.0.0.1 - - [25/Dec/2024:10:00:00 +0000] "GET /api/users HTTP/1.1" 200 1234 "-" "Mozilla/5.0"
192.168.1.50 - - [25/Dec/2024:10:00:01 +0000] "POST /api/orders HTTP/1.1" 201 5678 "https://example.com" "Mozilla/5.0"
10.0.0.1 - - [25/Dec/2024:10:00:02 +0000] "GET /api/products?category=electronics HTTP/1.1" 200 8901 "-" "curl/7.68.0"
```

### Stack Traces - Error Logs

```
Error: Connection failed
  at Database.connect (db.js:45:12)
  at UserService.getUser (user-service.js:23:8)
  at APIHandler.handleRequest (api-handler.js:67:15)
```

### Key-Value Logs

```
level=info message="User logged in" user_id=123 ip=192.168.1.100 duration_ms=45
level=error error_code=DB_001 message="Database connection failed" retry_count=3
cpu=45.2 memory=78.5 requests=1200 latency_ms=200
```

### Docker Container Logs

```
2024-12-25T10:00:00.123Z [INFO] Application started on port 3000
2024-12-25T10:00:01.456Z [ERROR] Failed to connect to database: connection timeout
2024-12-25T10:00:02.789Z [WARN] High memory usage: 85%
```

### Kubernetes Pod Logs

```
2024-12-25T10:00:00.123Z stdout F [INFO] Processing request from pod: frontend-abc-123
2024-12-25T10:00:01.456Z stderr F [ERROR] Health check failed: timeout
2024-12-25T10:00:02.789Z stdout F [INFO] Pod restarted successfully
```

### Firewall Logs

```
Dec 25 10:00:00 firewall kernel: [12345.678] IN=eth0 OUT= MAC=00:11:22:33:44:55 SRC=192.168.1.100 DST=10.0.0.1 LEN=60 TOS=0x00 PROTO=TCP SPT=54321 DPT=80
Dec 25 10:00:01 firewall kernel: [12346.789] IN=eth0 OUT= MAC=00:11:22:33:44:55 SRC=192.168.1.200 DST=10.0.0.1 LEN=60 TOS=0x00 PROTO=TCP SPT=54322 DPT=443
```

### Network Logs

```
10:00:00.123 IP 192.168.1.100.54321 > 10.0.0.1.80: Flags [S], seq 1234567890, win 65535
10:00:01.456 IP 10.0.0.1.80 > 192.168.1.100.54321: Flags [S.], seq 9876543210, ack 1234567891, win 65535
10:00:02.789 IP 192.168.1.100.54321 > 10.0.0.1.80: Flags [.], ack 9876543211, win 65535
```

### JSON-in-Text Logs

```
[2024-12-25 10:00:00] Application started with config: {"port": 3000, "env": "production", "debug": false}
[2024-12-25 10:00:01] User action: {"action": "login", "user_id": 123, "ip": "192.168.1.100"}
[2024-12-25 10:00:02] Metrics: {"cpu": 45, "memory": 78, "requests": 1200}
```

## SDK Usage Examples

### Node.js SDK

```javascript
const { VibexClient } = require('vibex-sdk');

const client = new VibexClient();

// JSON log with hybrid structure
await client.sendLog('json', {
  message: 'User logged in',
  level: 'info',
  metrics: { cpu: 45, memory: 78 },
  context: { user_id: 123, trace_id: 'abc-123' }
});

// Text log
await client.sendLog('text', 'Application started successfully');

// Web server log
await client.sendLog('web-server', '127.0.0.1 - - [25/Dec/2024:10:00:00 +0000] "GET /api/users HTTP/1.1" 200');

// Stack trace
await client.sendLog('stacktrace', 'Error: Connection failed\n  at file.js:10:5');
```

### Python SDK

```python
from vibex_sh import VibexClient

client = VibexClient()

# JSON log with hybrid structure
client.send_log('json', {
    'message': 'User logged in',
    'level': 'info',
    'metrics': {'cpu': 45, 'memory': 78},
    'context': {'user_id': 123, 'trace_id': 'abc-123'}
})

# Text log
client.send_log('text', 'Application started successfully')

# Web server log
client.send_log('web-server', '127.0.0.1 - - [25/Dec/2024:10:00:00 +0000] "GET /api/users HTTP/1.1" 200')

# Stack trace
client.send_log('stacktrace', 'Error: Connection failed\n  at file.py:10:5')
```

## Best Practices

### 1. Use Appropriate Log Types

- **JSON**: For structured application data, metrics, and events
- **Text**: For unstructured logs that need pattern detection
- **web-server**: When you know the log is from a web server
- **stacktrace**: For error stack traces
- **keyvalue**: For key-value formatted logs

### 2. Include Context for Tracing

Always include `trace_id`, `user_id`, or `request_id` in the `context` object to enable trace filtering:

```json
{
  "message": "Processing request",
  "context": {
    "trace_id": "trace-abc-123",
    "user_id": "u_999",
    "request_id": "req_456"
  }
}
```

### 3. Use Consistent Field Names

- **Metrics**: `cpu`, `memory`, `latency_ms`, `duration_ms`, `requests_per_sec`
- **Context**: `trace_id`, `user_id`, `request_id`, `correlation_id`, `span_id`, `session_id`
- **Levels**: `info`, `warn`, `error`, `debug`

### 4. Numeric Values Should Be Numbers

```json
// ✅ Good
{"cpu": 45, "memory": 78, "latency_ms": 200}

// ❌ Bad
{"cpu": "45", "memory": "78", "latency_ms": "200"}
```

### 5. Include Timestamps

Timestamps are auto-filled if missing, but it's better to include them:

```json
{"timestamp": "2024-01-15T10:00:00Z", "message": "Event occurred"}
```

### 6. Use Nested Objects for Complex Data

Nested objects are automatically extracted as facets:

```json
{
  "message": "Order processed",
  "order": {
    "id": "order_123",
    "total": 99.99,
    "items": 3
  },
  "customer": {
    "id": "cust_456",
    "tier": "premium"
  }
}
```

## When to Use Each Log Type

- **JSON**: Default choice for application logs, metrics, structured events
- **Text**: When you have unstructured logs or want automatic pattern detection
- **web-server**: For Nginx, Apache, or other web server access logs
- **loadbalancer**: For HAProxy, AWS ALB, or other load balancer logs
- **stacktrace**: For error stack traces from exceptions
- **firewall**: For iptables, pfSense, or Cisco ASA firewall logs
- **kubernetes**: For Kubernetes pod or container logs
- **docker**: For Docker container logs
- **network**: For tcpdump, wireshark, or other network packet logs
- **keyvalue**: For key-value pair formatted logs
- **json-in-text**: When JSON is embedded in text logs
- **smart-pattern**: For multi-language pattern matching
- **raw**: Fallback for any other log format

## Auto-Facet Detection

vibex.sh automatically detects and extracts:

- **Metrics**: Numeric fields become chartable metrics
- **Categorical Data**: String fields become filterable categories
- **Context Fields**: Known fields (trace_id, user_id, etc.) are indexed for filtering
- **Nested Objects**: Automatically extracted as facets

Following these rules ensures your logs are automatically parsed, visualized, and made searchable in vibex.sh dashboards.