Picoquic Shadow Plugin

Plugin Type: Service (IUT - Implementation Under Test)

Protocol: QUIC

Verified Source Location: plugins/services/iut/quic/picoquic_shadow/

Purpose and Overview

The Picoquic Shadow plugin provides a specialized integration of the Picoquic QUIC implementation designed specifically for use with the Shadow network simulator. This variant enables large-scale network simulation experiments with thousands of nodes while maintaining the performance characteristics and protocol compliance of the standard Picoquic implementation.

This implementation is valuable for:

• Large-scale network simulation: Simulating thousands of QUIC connections in Shadow
• Network topology research: Testing QUIC behavior across complex network topologies
• Performance analysis at scale: Evaluating QUIC performance under various network conditions
• Protocol research: Studying QUIC behavior in controlled, reproducible network environments
• Shadow-specific optimizations: Leveraging Shadow's simulation capabilities for QUIC research

The Shadow integration allows for deterministic, repeatable experiments with precise control over network conditions, making it ideal for academic research and large-scale protocol evaluation.

Key Features

Shadow Network Simulator Integration

• Deterministic simulation: Reproducible experiments with controlled network conditions
• Scalable architecture: Support for thousands of simulated nodes and connections
• Network topology simulation: Complex network topologies with realistic latency and bandwidth
• Event-driven simulation: Precise timing control for protocol analysis

Picoquic Core Features (Shadow-Compatible)

• QUIC version support: QUIC v1 (RFC 9000) and draft versions
• HTTP/3 support: Full HTTP/3 implementation over QUIC
• Connection migration: Seamless connection migration across network paths
• Loss recovery: Advanced loss detection and recovery mechanisms
• Congestion control: Multiple congestion control algorithms (NewReno, BBR, Cubic)

Research and Development Features

• Protocol experimentation: Support for custom QUIC extensions and modifications
• Performance metrics: Detailed simulation metrics and performance analysis
• Debugging support: Enhanced logging and state inspection for simulation environments
• Batch experimentation: Support for large-scale parameter sweeps and experiments

Inheritance Architecture

Inheritance-Based Implementation

The Picoquic Shadow plugin uses PANTHER's inheritance architecture, inheriting directly from BaseQUICServiceManager. This provides benefits through code reuse and consistent behavior while maintaining Shadow NS-specific integration patterns for deterministic network simulation.

Inheritance Architecture

The Picoquic Shadow implementation follows the template method pattern:

# panther/plugins/services/iut/quic/picoquic_shadow/picoquic_shadow.py
from panther.plugins.services.base.quic_service_base import BaseQUICServiceManager

class PicoquicShadowServiceManager(BaseQUICServiceManager):
    """Picoquic Shadow QUIC implementation with inheritance-based architecture."""

    def _get_implementation_name(self) -> str:
        return "picoquic_shadow"

    def _get_binary_name(self) -> str:
        return "picoquicdemo"  # Same as regular Picoquic

    # Customize for Shadow NS integration
    def _get_server_specific_args(self, **kwargs) -> List[str]:
        port = kwargs.get("port", 4443)
        # Shadow-specific socket binding
        return ["-p", str(port), "-L", "-l", "/tmp/server.log"]

    def _get_client_specific_args(self, **kwargs) -> List[str]:
        host = kwargs.get("host", "localhost")
        port = kwargs.get("port", 4443)
        # Shadow-specific connection patterns
        return [f"{host}", str(port), "-l", "/tmp/client.log"]

    # Shadow integration and deterministic testing handled by base class

Benefits of Inheritance Architecture

• Code Reduction: From extensive manual implementation to focused Shadow-specific patterns
• Consistent Behavior: Common QUIC functionality shared with all implementations
• Automatic Updates: New features in base class automatically available
• Event Integration: Built-in event emission for monitoring and debugging
• Shadow Compatibility: Maintains deterministic simulation requirements
• Research Focus: Allows focus on network simulation rather than infrastructure

What's Provided by Base Class

• Common Parameter Extraction: Port, host, certificate handling
• Standard Command Building: Template method pattern for command generation
• Event Emission: Service lifecycle and status events
• Error Handling: Timeout management and failure recovery
• Docker Integration: Standardized container build patterns
• Logging Integration: Structured logging with correlation tracking

What's Customized for Picoquic Shadow

• Binary Name: Uses standard picoquicdemo with Shadow adaptations
• Command Arguments: Shadow-specific logging and binding patterns
• Simulation Integration: Deterministic network simulation support
• Shadow Coordination: Integration with Shadow NS topology and timing

Deterministic Testing Benefits

• Reproducible Results: Identical network conditions across experiment runs
• Controlled Environment: Precise control over latency, bandwidth, and loss
• Scalable Simulation: Thousands of nodes without hardware limitations
• Event Coordination: Deterministic event ordering for protocol analysis

Requirements and Dependencies

Base Class Dependencies

The Picoquic Shadow implementation automatically inherits all base class dependencies, including the Command Processor, Event System, and common QUIC utilities. No additional setup is required for these core features.

System Requirements

• Shadow Network Simulator: Compatible version with QUIC support
• Build tools: CMake, GCC/Clang, Make
• Libraries: OpenSSL, libev, picotls
• Container runtime: Docker (for containerized Shadow experiments)

Python Dependencies

# Core dependencies from requirements
omegaconf>=2.0
dataclasses-json

Build Dependencies

The plugin automatically manages the following build dependencies:

• Picotls: TLS 1.3 implementation (configured via dependencies)
• OpenSSL: Cryptographic library
• CMake build system: For compilation management

Configuration Options

The Picoquic Shadow plugin accepts the following configuration parameters:

Parameter	Type	Default	Description
name	String	"picoquic_shadow"	Implementation identifier
type	ImplementationType	iut	Plugin type designation
shadow_compatible	Boolean	true	Shadow simulator compatibility flag
version.version	String	""	Picoquic version to use
version.commit	String	""	Git commit hash for specific version
version.dependencies	List	[]	Build dependencies configuration
version.client	Dict	{}	Client-specific configuration
version.server	Dict	{}	Server-specific configuration

Version Configuration Schema

version:
  version: "master"           # Git branch or tag
  commit: "abc123def456"      # Specific commit hash
  dependencies:               # Build dependencies
    - name: "picotls"
      url: "https://github.com/h2o/picotls.git"
      commit: "def789abc012"
  client:                     # Client-specific settings
    default_port: 4433
    max_connections: 1000
  server:                     # Server-specific settings
    bind_address: "0.0.0.0"
    port: 4433

Usage Examples

Basic Shadow Simulation Configuration

# experiment_config_shadow_quic.yaml
services:
  - name: "picoquic_shadow_server"
    type: "iut"
    implementation: "picoquic_shadow"
    role: "server"
    config:
      shadow_compatible: true
      version:
        version: "master"
        dependencies:
          - name: "picotls"
            url: "https://github.com/h2o/picotls.git"
            commit: "latest"

  - name: "picoquic_shadow_client"
    type: "iut"
    implementation: "picoquic_shadow"
    role: "client"
    config:
      shadow_compatible: true

Large-Scale Simulation Configuration

# Shadow topology with multiple clients
services:
  - name: "picoquic_shadow_server_central"
    type: "iut"
    implementation: "picoquic_shadow"
    role: "server"
    instances: 1
    config:
      shadow_compatible: true
      version:
        server:
          max_connections: 10000
          threads: 8

  - name: "picoquic_shadow_clients"
    type: "iut"
    implementation: "picoquic_shadow"
    role: "client"
    instances: 1000
    config:
      shadow_compatible: true
      version:
        client:
          connection_timeout: 30
          request_rate: 1.0

Research Configuration with Custom Extensions

# Research experiment with protocol modifications
services:
  - name: "picoquic_research_server"
    type: "iut"
    implementation: "picoquic_shadow"
    role: "server"
    config:
      shadow_compatible: true
      version:
        version: "research-branch"
        commit: "experimental-features"
        dependencies:
          - name: "picotls"
            url: "https://github.com/h2o/picotls.git"
            commit: "experimental"
        server:
          enable_custom_extensions: true
          log_level: "debug"
          metrics_collection: true

Integration Examples

Shadow Network Environment Integration

# Python integration with Shadow environment
from panther.plugins.plugin_loader import PluginLoader

# Load Shadow environment and Picoquic Shadow implementation
loader = PluginLoader()
shadow_env = loader.load_plugin('environments', 'network_environment', 'shadow_ns')
picoquic_shadow = loader.load_plugin('services', 'iut', 'quic', 'picoquic_shadow')

# Configure for large-scale simulation
shadow_config = {
    'topology': 'large_scale',
    'nodes': 1000,
    'simulation_time': 3600
}

picoquic_config = {
    'shadow_compatible': True,
    'version': {
        'version': 'master',
        'server': {'max_connections': 5000},
        'client': {'connection_rate': 10.0}
    }
}

# Initialize for simulation
shadow_env.initialize(shadow_config)
picoquic_shadow.initialize(picoquic_config)

Performance Analysis Setup

# Integration with performance monitoring
experiment_config = {
    'services': [
        {
            'name': 'picoquic_shadow_perf',
            'type': 'iut',
            'implementation': 'picoquic_shadow',
            'config': {
                'shadow_compatible': True,
                'version': {
                    'server': {
                        'enable_metrics': True,
                        'metrics_interval': 1.0,
                        'detailed_logging': True
                    }
                }
            }
        }
    ],
    'environments': [
        {
            'name': 'shadow_large_scale',
            'type': 'network_environment',
            'implementation': 'shadow_ns',
            'config': {
                'topology_file': 'large_scale_topology.xml',
                'simulation_time': 7200
            }
        }
    ]
}

Troubleshooting

Common Issues and Solutions

Shadow Compatibility Issues

# Verify Shadow simulator installation
shadow --version

# Check Picoquic Shadow build compatibility
docker logs <picoquic_shadow_container> | grep "Shadow"

# Validate Shadow-specific configurations
grep -r "shadow_compatible" /path/to/config/

Build and Dependency Issues

# Check dependency resolution
cat /opt/picoquic_shadow/dependencies.json

# Verify CMake configuration
cmake --version
cmake -LA /opt/picoquic_shadow/build/ | grep -i shadow

# Debug build process
docker build --no-cache -f Dockerfile --target debug .

Performance and Scaling Issues

# Monitor Shadow simulation performance
shadow --log-level debug --template-directory /etc/shadow

# Check resource utilization
docker stats <picoquic_shadow_container>

# Validate simulation parameters
shadow-validator --config shadow_config.xml

Debug Configuration

# Enhanced debugging for Shadow experiments
services:
  - name: "picoquic_shadow_debug"
    type: "iut"
    implementation: "picoquic_shadow"
    config:
      shadow_compatible: true
      version:
        server:
          log_level: "debug"
          enable_packet_logging: true
          enable_state_logging: true
          metrics_output: "/tmp/picoquic_metrics.log"
      debug:
        enable_gdb: false  # GDB not compatible with Shadow
        enable_strace: false  # Use Shadow's built-in tracing
        shadow_log_level: "debug"

Development and Extension

Building Custom Versions

# Build Picoquic Shadow with custom features
git clone https://github.com/private-octopus/picoquic.git picoquic_shadow
cd picoquic_shadow
git checkout <research-branch>

# Apply Shadow-specific patches
patch -p1 < shadow_integration.patch

# Build with Shadow support
mkdir build && cd build
cmake -DSHADOW_SUPPORT=ON -DCMAKE_BUILD_TYPE=Debug ..
make -j$(nproc)

Adding Custom Extensions

// Example custom extension for Shadow experiments
// filepath: picoquic_shadow_extensions.c

#ifdef SHADOW_SUPPORT
#include "shadow_interface.h"

// Custom metrics collection for Shadow
void picoquic_shadow_collect_metrics(picoquic_cnx_t* cnx) {
    if (shadow_is_simulation()) {
        shadow_log_metric("quic.rtt", cnx->path[0]->rtt_min);
        shadow_log_metric("quic.cwnd", cnx->path[0]->cwin);
        shadow_log_metric("quic.bytes_sent", cnx->nb_bytes_sent);
    }
}
#endif

Research Integration

# Integration with research frameworks
# filepath: picoquic_shadow_research.py

class PicoquicShadowResearch:
    def __init__(self, config):
        self.config = config
        self.shadow_interface = ShadowInterface()

    def run_parameter_sweep(self, parameters):
        """Run experiments with parameter variations"""
        results = []
        for param_set in parameters:
            shadow_config = self.generate_shadow_config(param_set)
            picoquic_config = self.generate_picoquic_config(param_set)

            result = self.run_experiment(shadow_config, picoquic_config)
            results.append(result)

        return self.analyze_results(results)

    def analyze_protocol_behavior(self, traces):
        """Analyze QUIC protocol behavior from Shadow traces"""
        metrics = {
            'connection_establishment_time': [],
            'handshake_duration': [],
            'migration_events': [],
            'congestion_window_evolution': []
        }

        for trace in traces:
            metrics.update(self.extract_metrics(trace))

        return self.generate_analysis_report(metrics)

Performance Considerations

Shadow Simulation Optimization

• Memory usage: Shadow simulations can be memory-intensive with many nodes
• CPU utilization: Multi-threaded Shadow simulations benefit from high core counts
• Simulation time: Large-scale experiments may require extended simulation periods
• Result storage: Simulation outputs can generate substantial data volumes

Network Topology Considerations

• Node scaling: Test with progressively larger node counts to identify scaling limits
• Bandwidth simulation: Ensure realistic bandwidth constraints for accurate results
• Latency modeling: Use appropriate latency distributions for target scenarios
• Loss patterns: Configure realistic packet loss patterns for protocol evaluation

Integration with Testing Framework

This plugin integrates with PANTHER's testing infrastructure through:

• Shadow environment plugins: Seamless integration with Shadow network simulator
• Protocol testing: Compatible with QUIC protocol test suites
• Performance monitoring: Integration with execution environment plugins
• Result analysis: Automated collection and analysis of simulation results

Related Documentation

• Shadow Network Environment Plugin
• Picoquic Standard Plugin
• QUIC Protocol Documentation
• Shadow Simulator Documentation
• Picoquic Project Repository

---

For additional support or questions about the Picoquic Shadow plugin, please refer to the main PANTHER documentation or contact the development team.