Example: Export and Analyze Data

This example shows how to export Renacer traces to JSON and CSV formats for programmatic analysis, automation, and data science workflows.

Overview

Renacer supports three export formats for programmatic analysis:

JSON - Structured data for scripts, APIs, and automation
CSV - Spreadsheet-friendly for Excel, R, Python pandas
HTML - Visual reports (covered in HTML Reports)

JSON Export

Basic JSON Export

Export syscall traces to JSON for programmatic analysis:

$ renacer --format json -- ls /tmp > trace.json

Output:

{
  "version": "0.4.1",
  "command": ["ls", "/tmp"],
  "syscalls": [
    {
      "name": "openat",
      "args": {
        "dirfd": "AT_FDCWD",
        "pathname": "/tmp",
        "flags": ["O_RDONLY", "O_DIRECTORY", "O_CLOEXEC"]
      },
      "return": {
        "value": 3,
        "error": null
      },
      "timestamp": 1234567890.123456,
      "duration_ns": 12345,
      "pid": 12345
    },
    {
      "name": "getdents64",
      "args": {
        "fd": 3,
        "count": 32768
      },
      "return": {
        "value": 1024,
        "error": null
      },
      "timestamp": 1234567890.234567,
      "duration_ns": 5678,
      "pid": 12345
    }
  ],
  "summary": {
    "total_syscalls": 45,
    "total_duration_ms": 123.456,
    "error_count": 2
  }
}

JSON Structure:

version - Renacer version (for compatibility)
command - Traced command and arguments
syscalls[] - Array of syscall events
summary - Aggregate statistics

JSON with Statistics

Combine with statistics mode for aggregate data:

$ renacer --format json -c -- ./myapp > stats.json

Additional fields in JSON:

{
  "statistics": {
    "read": {
      "calls": 1000,
      "errors": 0,
      "total_time_ms": 123.456,
      "avg_time_ms": 0.123,
      "min_time_ms": 0.001,
      "max_time_ms": 5.678,
      "p50_ms": 0.100,
      "p90_ms": 0.300,
      "p99_ms": 1.234
    }
  }
}

Processing JSON with jq

Extract Specific Data

Example 1: List all syscall names

$ jq -r '.syscalls[].name' trace.json | sort | uniq

Output:

close
getdents64
openat
read
write

Example 2: Find all errors

$ jq '.syscalls[] | select(.return.error != null)' trace.json

Output:

{
  "name": "openat",
  "args": {
    "pathname": "/nonexistent"
  },
  "return": {
    "value": -1,
    "error": "ENOENT"
  }
}

Example 3: Count syscalls by name

$ jq '.syscalls | group_by(.name) | map({name: .[0].name, count: length})' trace.json

Output:

[
  {"name": "read", "count": 1000},
  {"name": "write", "count": 500},
  {"name": "open", "count": 100}
]

Calculate Aggregate Statistics

Example 1: Total time by syscall

$ jq '.syscalls | group_by(.name) | map({
    name: .[0].name,
    total_ns: (map(.duration_ns) | add),
    total_ms: ((map(.duration_ns) | add) / 1000000)
  }) | sort_by(.total_ms) | reverse' trace.json

Output:

[
  {"name": "read", "total_ns": 123456789, "total_ms": 123.456},
  {"name": "write", "total_ns": 98765432, "total_ms": 98.765},
  {"name": "open", "total_ns": 45678901, "total_ms": 45.678}
]

Example 2: Average latency per syscall

$ jq '.syscalls | group_by(.name) | map({
    name: .[0].name,
    avg_ns: ((map(.duration_ns) | add) / length),
    avg_ms: (((map(.duration_ns) | add) / length) / 1000000)
  })' trace.json

Example 3: Find slowest syscalls

$ jq '.syscalls | sort_by(.duration_ns) | reverse | .[0:10] | .[] | {name, duration_ms: (.duration_ns / 1000000)}' trace.json

Output:

{"name": "fsync", "duration_ms": 23.456}
{"name": "read", "duration_ms": 12.345}
{"name": "write", "duration_ms": 9.876}
...

Filter and Transform

Example 1: Extract file operations only

$ jq '.syscalls[] | select(.name | test("^(open|read|write|close)"))'  trace.json

Example 2: Convert timestamps to readable format

$ jq '.syscalls[] | {
    name,
    time: (.timestamp | strftime("%Y-%m-%d %H:%M:%S")),
    duration_ms: (.duration_ns / 1000000)
  }' trace.json

Example 3: Group errors by type

$ jq '.syscalls | map(select(.return.error != null)) | group_by(.return.error) | map({
    error: .[0].return.error,
    count: length,
    syscalls: (map(.name) | unique)
  })' trace.json

Output:

[
  {
    "error": "ENOENT",
    "count": 15,
    "syscalls": ["open", "stat"]
  },
  {
    "error": "EACCES",
    "count": 3,
    "syscalls": ["open"]
  }
]

CSV Export

Basic CSV Export

Export for spreadsheet analysis:

$ renacer --format csv -- ls /tmp > trace.csv

Output:

name,args,return_value,return_error,timestamp,duration_ns,pid,source_file,source_line,source_function
openat,"dirfd=AT_FDCWD pathname=/tmp flags=O_RDONLY|O_DIRECTORY|O_CLOEXEC",3,,1234567890.123456,12345,12345,,,
getdents64,"fd=3 count=32768",1024,,1234567890.234567,5678,12345,,,
close,"fd=3",0,,1234567890.345678,1234,12345,,,

Column descriptions:

name - Syscall name
args - Space-separated arguments
return_value - Return value (integer)
return_error - Error code (if any)
timestamp - Unix timestamp with microseconds
duration_ns - Duration in nanoseconds
pid - Process ID
source_file - Source file (with --source)
source_line - Line number (with --source)
source_function - Function name (with --source)

CSV with Statistics

$ renacer --format csv -c -- ./myapp > stats.csv

Statistics CSV format:

syscall,calls,errors,total_time_ms,avg_time_ms,min_time_ms,max_time_ms,p50_ms,p90_ms,p99_ms
read,1000,0,123.456,0.123,0.001,5.678,0.100,0.300,1.234
write,500,0,98.765,0.197,0.005,8.901,0.150,0.450,2.345
open,100,2,45.678,0.456,0.010,12.345,0.400,1.200,5.678

Processing CSV with Command-Line Tools

Using csvkit

Example 1: View summary statistics

$ csvstat trace.csv

Output:

Column: name
  Unique values: 10
  Most common: read (1000x)

Column: duration_ns
  Mean: 12345.67
  Median: 5678.0
  Max: 123456.0

Example 2: Filter to errors only

$ csvgrep -c return_error -r '.+' trace.csv

Example 3: Sort by duration

$ csvsort -c duration_ns -r trace.csv | head -20

Example 4: Select specific columns

$ csvcut -c name,duration_ns,return_error trace.csv

Using awk

Example 1: Calculate average duration per syscall

$ tail -n +2 trace.csv | awk -F',' '{
    sum[$1] += $6;  # duration_ns column
    count[$1]++;
  }
  END {
    for (name in sum) {
      printf "%s: avg %.2f us\n", name, sum[name]/count[name]/1000
    }
  }' | sort -k2 -rn

Output:

fsync: avg 4567.89 us
read: avg 123.45 us
write: avg 98.76 us

Example 2: Count errors by type

$ tail -n +2 trace.csv | awk -F',' '$4 != "" {errors[$4]++} END {for (e in errors) print e, errors[e]}' | sort -k2 -rn

Output:

ENOENT 15
EACCES 3
EINVAL 1

Analysis with Python pandas

Load and Explore

import pandas as pd
import numpy as np

# Load trace
df = pd.read_csv('trace.csv')

# Basic info
print(df.info())
print(df.describe())

# First few rows
print(df.head())

Aggregate Analysis

Example 1: Group by syscall name

# Group by syscall, calculate statistics
stats = df.groupby('name').agg({
    'duration_ns': ['count', 'mean', 'std', 'min', 'max'],
    'return_error': 'count'
}).round(2)

print(stats.sort_values(('duration_ns', 'mean'), ascending=False))

Output:

            duration_ns                              return_error
                  count      mean       std    min        max        count
name
fsync             100  4567.89  1234.56  1000  12345.00       100
read             1000   123.45    45.67    10   5678.00      1000
write             500    98.76    34.56    20   8901.00       500

Example 2: Time series analysis

# Convert timestamp to datetime
df['time'] = pd.to_datetime(df['timestamp'], unit='s')

# Resample to 1-second bins
time_series = df.set_index('time').resample('1S')['duration_ns'].agg(['count', 'sum', 'mean'])

print(time_series)

Example 3: Error analysis

# Filter to errors only
errors = df[df['return_error'].notna()]

# Group errors by type and syscall
error_summary = errors.groupby(['return_error', 'name']).size().unstack(fill_value=0)

print(error_summary)

Output:

            name  close  open  read  stat
return_error
EACCES              0     3     0     0
ENOENT              0    10     0     5

Visualization

Example 1: Duration distribution

import matplotlib.pyplot as plt

# Convert to milliseconds
df['duration_ms'] = df['duration_ns'] / 1_000_000

# Histogram
df.boxplot(column='duration_ms', by='name', figsize=(12, 6))
plt.ylabel('Duration (ms)')
plt.title('Syscall Duration Distribution')
plt.savefig('duration-boxplot.png')

Example 2: Top syscalls by time

# Calculate total time per syscall
total_time = df.groupby('name')['duration_ns'].sum().sort_values(ascending=False).head(10)

# Bar chart
total_time.plot(kind='barh', figsize=(10, 6))
plt.xlabel('Total Time (ns)')
plt.title('Top 10 Syscalls by Total Time')
plt.tight_layout()
plt.savefig('top-syscalls.png')

Example 3: Timeline plot

# Set timestamp as index
df['time'] = pd.to_datetime(df['timestamp'], unit='s')
df = df.set_index('time')

# Plot syscall rate over time
df['name'].resample('100ms').count().plot(figsize=(12, 4))
plt.ylabel('Syscalls per 100ms')
plt.title('Syscall Rate Over Time')
plt.savefig('timeline.png')

Analysis with R

Load and Summarize

library(dplyr)
library(ggplot2)

# Load data
trace <- read.csv('trace.csv')

# Summary statistics
summary(trace)

# Group by syscall
syscall_stats <- trace %>%
  group_by(name) %>%
  summarise(
    count = n(),
    avg_duration = mean(duration_ns),
    max_duration = max(duration_ns),
    errors = sum(!is.na(return_error))
  ) %>%
  arrange(desc(avg_duration))

print(syscall_stats)

Visualization

Example 1: Duration boxplot

ggplot(trace, aes(x = name, y = duration_ns / 1000)) +
  geom_boxplot() +
  coord_flip() +
  labs(
    title = "Syscall Duration Distribution",
    x = "Syscall",
    y = "Duration (microseconds)"
  ) +
  theme_minimal()

ggsave("r-duration-boxplot.png", width = 10, height = 6)

Example 2: Time series

trace$time <- as.POSIXct(trace$timestamp, origin = "1970-01-01")

ggplot(trace, aes(x = time)) +
  geom_histogram(bins = 50) +
  labs(
    title = "Syscall Frequency Over Time",
    x = "Time",
    y = "Count"
  ) +
  theme_minimal()

CI/CD Integration

Automated Performance Regression Detection

GitHub Actions example:

name: Performance Check

on: [pull_request]

jobs:
  perf-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Build Release
        run: cargo build --release

      - name: Baseline Performance
        run: |
          git checkout main
          cargo build --release
          renacer --format json -c -- ./target/release/myapp > baseline.json

      - name: PR Performance
        run: |
          git checkout ${{ github.head_ref }}
          cargo build --release
          renacer --format json -c -- ./target/release/myapp > pr.json

      - name: Compare Performance
        run: |
          jq -r '.summary.total_duration_ms' baseline.json > baseline_time.txt
          jq -r '.summary.total_duration_ms' pr.json > pr_time.txt

          BASELINE=$(cat baseline_time.txt)
          PR=$(cat pr_time.txt)
          THRESHOLD=10  # 10% regression threshold

          DIFF=$(echo "scale=2; ($PR - $BASELINE) / $BASELINE * 100" | bc)

          if (( $(echo "$DIFF > $THRESHOLD" | bc -l) )); then
            echo "❌ Performance regression: ${DIFF}% slower"
            exit 1
          else
            echo "✅ Performance acceptable: ${DIFF}% change"
          fi

Monitoring Integration

Export to Prometheus:

#!/bin/bash
# Export Renacer stats to Prometheus format

renacer --format json -c -- ./production-app > trace.json

jq -r '.statistics | to_entries[] | "syscall_duration_seconds{\(.key)} \(.value.total_time_ms / 1000)"' trace.json > metrics.prom

# Push to Prometheus pushgateway
curl -X POST --data-binary @metrics.prom http://pushgateway:9091/metrics/job/app_trace

Result:

syscall_duration_seconds{read} 0.123456
syscall_duration_seconds{write} 0.098765
syscall_duration_seconds{fsync} 0.045678

Best Practices

1. Use Appropriate Format

# JSON for automation/scripting
renacer --format json -c -- ./app > stats.json

# CSV for spreadsheet/data science
renacer --format csv -c -- ./app > stats.csv

# HTML for sharing with team
renacer --format html -c -- ./app > report.html

2. Compress Large Exports

# Compress JSON
renacer --format json -- ./app | gzip > trace.json.gz

# Analyze without decompressing
zcat trace.json.gz | jq '.summary'

3. Filter Before Export

# Only export file operations
renacer --format json -e 'trace=file' -- ./app > file-ops.json

# Smaller file, faster processing

4. Combine with Statistics Mode

# Full trace (large)
renacer --format json -- ./app > full-trace.json

# Summary only (small)
renacer --format json -c -- ./app > summary.json

5. Version Your Exports

# Include version in filename
renacer --format json -c -- ./app > "trace-$(git describe --tags)-$(date +%Y%m%d).json"

Troubleshooting

Large Export Files

Problem: JSON/CSV exports are gigabytes in size.

Solutions:

Filter syscalls:

renacer --format json -e 'trace=file' -- ./app

Use statistics mode:

renacer --format json -c -- ./app  # Summary only

Compress:

renacer --format json -- ./app | gzip > trace.json.gz

Stream processing:

renacer --format json -- ./app | jq -c '.syscalls[] | select(.name == "read")'

JSON Parsing Errors

Problem: jq reports syntax error.

Causes:

Incomplete export (process interrupted)
Corrupted file

Solution:

# Verify JSON is complete
tail -1 trace.json  # Should show closing }

# Validate JSON
jq empty trace.json && echo "Valid JSON" || echo "Invalid JSON"

# Re-export if corrupted

CSV Encoding Issues

Problem: Excel shows garbled characters.

Solution:

# Add UTF-8 BOM for Excel
renacer --format csv -- ./app | iconv -f UTF-8 -t UTF-8-BOM > trace.csv

Summary

Export formats for different use cases:

Format	Best For	Tools
JSON	Automation, scripts, APIs	jq, Python, JavaScript
CSV	Spreadsheets, data science	Excel, R, pandas, csvkit
HTML	Sharing, documentation	Browser (any device)

Common workflows:

✅ jq - Command-line JSON processing
✅ csvkit - CSV analysis (csvstat, csvgrep, csvsort)
✅ pandas - Python data analysis and visualization
✅ R - Statistical analysis and plotting
✅ CI/CD - Automated performance regression detection
✅ Monitoring - Export to Prometheus/Grafana

Key practices:

Filter syscalls before export (reduce file size)
Use statistics mode for summaries
Compress large exports (gzip)
Version exported files (Git tags + timestamps)
Validate exports (jq empty, csvstat)

JSON Output Format - JSON specification
CSV Output Format - CSV specification
Statistics Mode - Aggregate data
Filtering Syscalls - Reduce export size

The Renacer Book - Pure Rust System Call Tracing