Error Analysis Patterns

This guide provides comprehensive strategies for analyzing, interpreting, and acting on different types of errors discovered during Dochia API testing. Learn how to identify patterns, prioritize issues, and implement effective remediation strategies.

Overview

Effective error analysis involves:

• Understanding different error categories and their implications
• Identifying patterns in error occurrences
• Prioritizing issues based on security and business impact
• Implementing systematic remediation approaches
• Establishing monitoring and prevention strategies

Error Categories

Security Vulnerabilities

SQL Injection Indicators

Detection Patterns:

# Look for database error leaks
dochia test --contract api.yml --server https://api.example.com \
  --error-keywords sql-injection-keywords.properties

# sql-injection-keywords.properties
SELECT
INSERT
UPDATE
DELETE
DROP
CREATE
ALTER
UNION
mysql_
postgresql
ORA-
SQLite
syntax error
near "
quoted string

Analysis Approach:

• High Priority: Direct SQL error messages in responses
• Medium Priority: Timing differences indicating blind SQL injection
• Low Priority: Generic database connection errors

Remediation Steps:

1. Implement parameterized queries
2. Add input validation and sanitization
3. Apply principle of least privilege to database users
4. Enable database query logging for monitoring

Authentication & Authorization Flaws

Detection Patterns:

# Test authentication bypass scenarios
dochia test --contract api.yml --server https://api.example.com \
  --playbooks BypassAuthenticationPlaybook \
  --paths /api/v1/admin,/api/v1/users

Common Error Indicators:

• 401 Unauthorized: Missing or invalid authentication
• 403 Forbidden: Insufficient permissions
• 200 OK with sensitive data: Authorization bypass
• 500 Internal Server Error: Authentication system failures

Analysis Framework:

# Analyze authentication patterns (Free version - process individual reports)
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code == 200 and .path | contains("admin")) |
    "CRITICAL: Admin endpoint accessible without auth: \(.path)"' "$report"
done

# Check for privilege escalation
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code == 200 and .playbook | contains("BypassAuthenticationPlaybook")) |
    "WARNING: Potential privilege escalation: \(.path) - \(.playbook)"' "$report"
done

Information Disclosure

Detection Patterns:

# Look for information leakage
dochia test --contract api.yml --server https://api.example.com \
  --error-keywords info-disclosure-keywords.properties

# info-disclosure-keywords.properties
Exception
Error
Traceback
at java
at com.
password
token
key
secret
debug
trace
verbose
dump
stack trace
internal error

Severity Assessment:

• Critical: Credentials, tokens, or keys in responses
• High: Stack traces revealing internal architecture
• Medium: Debug information exposing business logic
• Low: Generic error messages with minimal information

Input Validation Issues

Data Type Violations

Analysis Pattern:

# Analyze type-related errors (Free version approach)
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.playbook | contains("InvalidValuesInEnumsFieldsPlaybook")) |
    {
      path: .path,
      field: .field_name,
      expected: .expected_type,
      provided: .test_value,
      response: .status_code
    }' "$report"
done | jq -s 'group_by(.path) | map({path: .[0].path, issues: length})'

Common Patterns:

• String in numeric fields: 400 Bad Request expected
• Invalid enum values: 422 Unprocessable Entity expected
• Null in required fields: 400 Bad Request expected
• Oversized strings: 413 Payload Too Large expected

Boundary Value Problems

Detection Strategy:

# Test boundary conditions
dochia test --contract api.yml --server https://api.example.com \
  --playbooks VeryLargeStringsInFieldsPlaybook,VeryLargeIntegersInNumericFieldsPlaybook,StringFieldsLeftBoundaryPlaybook,StringFieldsRightBoundaryPlaybook

Analysis Approach:

# Identify boundary issues
find ./reports -name "*.json" | while read report; do
  jq -r 'select((.playbook | contains("VeryLargeStringsInFieldsPlaybook") or contains("VeryLargeIntegersInNumericFieldsPlaybook")) and .status_code == 500) |
    "ISSUE: \(.path) field \(.field_name) causes server error with large values"' "$report"
done

Performance and Reliability Issues

Timeout and Resource Exhaustion

Detection Patterns:

# Monitor response times and timeouts
dochia test --contract api.yml --server https://api.example.com \
  --max-response-time 5000 \
  --detailed-execution-stats

Analysis Metrics:

# Analyze performance patterns
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.response_time_ms > 5000) |
    {
      path: .path,
      method: .http_method,
      response_time: .response_time_ms,
      playbook: .playbook
    }' "$report"
done | jq -s 'sort_by(.response_time) | reverse'

Rate Limiting and Throttling

Detection Strategy:

# Test rate limiting behavior
dochia test --contract api.yml --server https://api.example.com \
  --max-requests-per-minute 1000 \
  --playbooks LargeNumberOfRandomHeadersPlaybook,LargeNumberOfRandomAlphanumericHeadersPlaybook

Expected Responses:

• 429 Too Many Requests: Proper rate limiting
• 503 Service Unavailable: Server overload
• 200 OK with degraded performance: Insufficient rate limiting

Error Pattern Analysis

Frequency Analysis

High-Frequency Errors:

# Identify most common error patterns
find ./reports -name "*.json" -exec jq -r '.status_code' {} \; | sort | uniq -c | sort -nr

Path-Specific Analysis:

# Errors by API path
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code >= 400) |
    "\(.path): \(.status_code)"' "$report"
done | sort | uniq -c | sort -nr

Correlation Analysis

Playbook Correlation:

# Errors by playbook type
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code >= 400) |
    "\(.playbook): \(.status_code)"' "$report"
done | sort | uniq -c | sort -nr

Field-Specific Issues:

# Field-level error patterns
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code >= 400 and .field_name) |
    "\(.field_name) (\(.field_type)): \(.status_code)"' "$report"
done | sort | uniq -c | sort -nr

Prioritization Framework

Risk Assessment Matrix

Critical Priority (Fix Immediately):

• SQL injection vulnerabilities
• Authentication bypass
• Sensitive data exposure
• Remote code execution indicators

High Priority (Fix Within 24 Hours):

• Authorization flaws
• Input validation bypasses
• Information disclosure
• Denial of service vulnerabilities

Medium Priority (Fix Within Week):

• Improper error handling
• Performance degradation
• Rate limiting issues
• Data validation inconsistencies

Low Priority (Fix in Next Release):

• Minor information leakage
• Non-critical validation errors
• Performance optimization opportunities
• Documentation inconsistencies

Business Impact Assessment

Revenue Impact:

# Identify errors in critical business paths
CRITICAL_PATHS="/api/v1/orders,/api/v1/payments,/api/v1/checkout"

find ./reports -name "*.json" | while read report; do
  for path in $(echo "$CRITICAL_PATHS" | tr ',' ' '); do
    jq -r --arg path "$path" 'select(.status_code >= 400 and (.path | contains($path))) |
      "BUSINESS CRITICAL: \(.path) - \(.status_code) - \(.playbook)"' "$report"
  done
done

User Experience Impact:

# Analyze user-facing endpoint errors
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code >= 400 and (.path | contains("/users") or contains("/profile"))) |
    "UX IMPACT: \(.path) - \(.status_code)"' "$report"
done | sort | uniq -c

Remediation Strategies

Systematic Approach

1. Immediate Response (0-4 hours):

#!/bin/bash
# emergency-response.sh

# Identify critical security issues
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code == 200 and (.response_body | contains("SELECT") or contains("password") or contains("token"))) |
    "EMERGENCY: \(.path) - Potential data leak"' "$report"
done

# Check for authentication bypasses
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code == 200 and .path | contains("admin") and .playbook | contains("BypassAuthenticationPlaybook")) |
    "EMERGENCY: \(.path) - Authentication bypass"' "$report"
done

2. Short-term Fixes (1-7 days):

• Input validation improvements
• Error message sanitization
• Rate limiting implementation
• Authentication strengthening

3. Long-term Improvements (1-4 weeks):

• Architecture security reviews
• Comprehensive testing integration
• Security training for development teams
• Automated security scanning in CI/CD

Code-Level Remediation

Input Validation Patterns:

// Before: Vulnerable to injection
String query = "SELECT * FROM users WHERE id = " + userId;

// After: Parameterized query
String query = "SELECT * FROM users WHERE id = ?";
PreparedStatement stmt = connection.prepareStatement(query);
stmt.setString(1, userId);

Error Handling Patterns:

// Before: Information disclosure
catch (SQLException e) {
    return ResponseEntity.status(500).body("Error: " + e.getMessage());
}

// After: Sanitized error response
catch (SQLException e) {
    logger.error("Database error for user {}: {}", userId, e.getMessage());
    return ResponseEntity.status(500).body("Internal server error");
}

Monitoring and Prevention

Continuous Monitoring

Automated Error Detection:

#!/bin/bash
# continuous-monitoring.sh

# Run daily security scans
dochia test --contract api.yml --server https://api.example.com \
  --playbooks BypassAuthenticationPlaybook,CheckSecurityHeadersPlaybook,InvalidValuesInEnumsFieldsPlaybook \
  --output ./daily-security-scan \
  --timestamp-reports

# Alert on critical findings
CRITICAL_COUNT=0
find ./daily-security-scan -name "*.json" | while read report; do
  COUNT=$(jq -r 'select(.status_code == 200 and (.response_body | contains("SELECT") or contains("Exception")))' "$report" | wc -l)
  CRITICAL_COUNT=$((CRITICAL_COUNT + COUNT))
done

if [ "$CRITICAL_COUNT" -gt 0 ]; then
  echo "ALERT: $CRITICAL_COUNT critical security issues found"
  # Send alert to security team
fi

Trend Analysis:

# Weekly trend analysis
find ./reports -name "*.json" -mtime -7 | while read report; do
  jq -r '{
    date: (.timestamp // now | strftime("%Y-%m-%d")),
    path: .path,
    status: .status_code,
    severity: (if .status_code >= 500 then "high" elif .status_code >= 400 then "medium" else "low" end)
  }' "$report"
done | jq -s 'group_by(.date) | map({date: .[0].date, high: map(select(.severity == "high")) | length})'

Prevention Strategies

Development Integration:

# .github/workflows/security-testing.yml
name: Security Testing
on: [push, pull_request]
jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run Dochia Security Tests
        run: |
          dochia test --contract openapi.yml --server ${{ secrets.STAGING_URL }} \
            --playbooks BypassAuthenticationPlaybook,CheckSecurityHeadersPlaybook,InvalidValuesInEnumsFieldsPlaybook,EmptyStringsInFieldsPlaybook,NullValuesInFieldsPlaybook \
            --output ./security-results
          
          # Fail build on critical issues
          CRITICAL=0
          find ./security-results -name "*.json" | while read report; do
            COUNT=$(jq -r 'select(.status_code == 200 and (.response_body | contains("SELECT") or contains("password")))' "$report" | wc -l)
            CRITICAL=$((CRITICAL + COUNT))
          done
          
          if [ "$CRITICAL" -gt 0 ]; then
            echo "Build failed: $CRITICAL critical security issues found"
            exit 1
          fi

Training and Awareness:

• Regular security training sessions
• Code review checklists including security patterns
• Incident response procedures
• Security testing methodology documentation

Reporting and Communication

Executive Summary Template

# Security Testing Summary - [Date]

## Critical Findings: [X]
- [Brief description of critical issues]
- Immediate action required

## High Priority Issues: [X]
- [Summary of high priority findings]
- Target resolution: 24-48 hours

## Trends and Patterns:
- [Analysis of recurring issues]
- [Improvement recommendations]

## Risk Assessment:
- Business Impact: [High/Medium/Low]
- Technical Complexity: [High/Medium/Low]
- Recommended Timeline: [Specific dates]

Technical Report Generation

#!/bin/bash
# generate-security-report.sh

REPORT_DATE=$(date +%Y-%m-%d)
REPORT_FILE="security-report-${REPORT_DATE}.md"

echo "# Security Analysis Report - ${REPORT_DATE}" > "$REPORT_FILE"
echo "" >> "$REPORT_FILE"

# Critical issues
echo "## Critical Issues" >> "$REPORT_FILE"
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code == 200 and (.response_body | contains("SELECT") or contains("password"))) |
    "- **\(.path)**: \(.playbook) - Potential data exposure"' "$report"
done >> "$REPORT_FILE"

# Summary statistics
echo "" >> "$REPORT_FILE"
echo "## Summary Statistics" >> "$REPORT_FILE"
echo "- Total Tests: $(find ./reports -name "*.json" | wc -l)" >> "$REPORT_FILE"
echo "- Failed Tests: $(find ./reports -name "*.json" -exec jq -r 'select(.status_code >= 400)' {} \; | wc -l)" >> "$REPORT_FILE"
echo "- Critical Issues: $(find ./reports -name "*.json" -exec jq -r 'select(.status_code == 200 and (.response_body | contains("SELECT")))' {} \; | wc -l)" >> "$REPORT_FILE"

Best Practices

Analysis Workflow

1. Immediate Triage (0-30 minutes)

   • Identify critical security vulnerabilities
   • Check for authentication bypasses
   • Look for sensitive data exposure

2. Detailed Analysis (30 minutes - 2 hours)

   • Categorize all findings by severity
   • Analyze patterns and correlations
   • Assess business impact

3. Remediation Planning (2-4 hours)

   • Prioritize fixes based on risk assessment
   • Assign ownership and timelines
   • Plan verification testing

4. Follow-up and Monitoring (Ongoing)

   • Verify fixes are effective
   • Monitor for regression
   • Update testing strategies

Tool Integration

SIEM Integration:

# Send findings to SIEM system
find ./reports -name "*.json" | while read report; do
  jq -c 'select(.status_code >= 400) |
    {
      timestamp: now,
      source: "dochia",
      severity: (if .status_code >= 500 then "high" else "medium" end),
      path: .path,
      method: .http_method,
      status_code: .status_code,
      playbook: .playbook
    }' "$report" | while read event; do
    curl -X POST "$SIEM_ENDPOINT" \
      -H "Authorization: Bearer $SIEM_TOKEN" \
      -H "Content-Type: application/json" \
      -d "$event"
  done
done

Ticketing System Integration:

# Create tickets for high-priority issues
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code >= 400 and (.response_body | contains("Exception") or contains("Error"))) |
    {
      title: "Security Issue: \(.path)",
      description: "Playbook: \(.playbook)\nStatus: \(.status_code)\nPath: \(.path)",
      priority: (if .status_code >= 500 then "high" else "medium" end)
    }' "$report" | while read ticket; do
    # Create ticket in your system
    echo "$ticket" | curl -X POST "$TICKET_SYSTEM_API" \
      -H "Authorization: Bearer $TICKET_TOKEN" \
      -H "Content-Type: application/json" \
      -d @-
  done
done

Common Error Patterns

HTTP Status Code Analysis

2xx Success Codes (Potential Issues):

• 200 OK with error content: Check response body for leaked information
• 201 Created without proper validation: May indicate insufficient input validation
• 204 No Content for data operations: Verify if response should contain data

4xx Client Error Codes (Expected):

• 400 Bad Request: Proper input validation working
• 401 Unauthorized: Authentication system functioning
• 403 Forbidden: Authorization controls in place
• 404 Not Found: Resource protection working
• 422 Unprocessable Entity: Business logic validation active

5xx Server Error Codes (Investigate):

• 500 Internal Server Error: Potential application vulnerabilities
• 502 Bad Gateway: Infrastructure or dependency issues
• 503 Service Unavailable: Resource exhaustion or DoS vulnerability
• 504 Gateway Timeout: Performance or availability issues

Response Content Analysis

Information Leakage Indicators:

# Check for common information disclosure patterns
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.response_body | contains("Exception") or contains("at java") or contains("SQLException")) |
    "INFO LEAK: \(.path) - \(.status_code) - Stack trace in response"' "$report"
done

Authentication Bypass Patterns:

# Detect potential authentication bypasses
find ./reports -name "*.json" | while read report; do
  jq -r 'select(.status_code == 200 and (.path | contains("admin") or contains("internal")) and (.playbook | contains("BypassAuthenticationPlaybook"))) |
    "AUTH BYPASS: \(.path) - Admin/Internal endpoint accessible"' "$report"
done

See Also

• Best Practices - General testing best practices
• Performance Tuning - Performance optimization strategies
• Enterprise Deployment - Enterprise-scale deployment patterns
• CI/CD Integration - Automated testing pipelines