Service Discovery & Auto-Detection

Nodewarden's intelligent service discovery automatically identifies and monitors services running on your infrastructure, reducing manual configuration and ensuring comprehensive coverage.

How Service Discovery Works

Automatic Detection Process

The Nodewarden agent continuously scans your systems to discover:

1. Process Analysis: Examines running processes and command lines
2. Port Scanning: Identifies listening services on common ports
3. Configuration Files: Reads service configuration when accessible
4. Environment Variables: Extracts service-specific settings
5. Health Endpoints: Tests for common health check URLs

Discovery Methods

Process-based Discovery

Identifies services by analyzing running processes:

bashCopy
# Example: MySQL Detection
Process: mysqld
Port: 3306
Config: /etc/mysql/my.cnf
Version: 8.0.28
Status: Running

Network-based Discovery

Scans for services listening on network ports:

bashCopy
# Port scan results
Port 80: nginx (HTTP)
Port 443: nginx (HTTPS) 
Port 3306: mysql (Database)
Port 6379: redis (Cache)
Port 5432: postgresql (Database)

Configuration-based Discovery

Reads service configuration files for detailed information:

yamlCopy
# Apache Configuration Discovery
Service: Apache HTTP Server
Version: 2.4.41
DocumentRoot: /var/www/html
VirtualHosts:
  - example.com (:80)
  - api.example.com (:443)
Modules: [ssl, rewrite, proxy]

Supported Services

Web Servers

Apache HTTP Server

Detection: Process name httpd or apache2 Metrics Collected:

• Request rate and response times
• Active connections and workers
• Virtual host statistics
• Module status and configuration
• SSL certificate information

Configuration Discovery:

apacheCopy
# Detected from /etc/apache2/apache2.conf
ServerRoot /etc/apache2
Listen 80
Listen 443 ssl
LoadModule ssl_module modules/mod_ssl.so

Nginx

Detection: Process name nginx Metrics Collected:

• Request processing statistics
• Connection handling metrics
• Upstream server health
• Cache hit/miss ratios
• SSL/TLS metrics

Configuration Discovery:

nginxCopy
# Detected from /etc/nginx/nginx.conf
worker_processes auto;
upstream backend {
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
}

IIS (Windows)

Detection: Process name w3wp.exe Metrics Collected:

• Application pool status
• Request processing metrics
• Connection statistics
• ASP.NET performance counters

Databases

MySQL/MariaDB

Detection: Process mysqld on port 3306 Metrics Collected:

• Connection statistics
• Query performance metrics
• Buffer pool utilization
• Replication status
• Table and index statistics

Configuration Discovery:

iniCopy
# Detected from /etc/mysql/my.cnf
[mysqld]
port = 3306
bind-address = 0.0.0.0
max_connections = 1000
innodb_buffer_pool_size = 2G

Automatic Monitoring Setup:

yamlCopy
# Auto-generated monitoring configuration
Service: MySQL
Host: db-server-01
Port: 3306
Authentication: Auto-discovered from .my.cnf
Metrics:
  - Connections (active, max, usage)
  - Queries (rate, slow queries)
  - InnoDB (buffer pool, lock waits)
  - Replication (lag, status)

PostgreSQL

Detection: Process postgres on port 5432 Metrics Collected:

• Connection and session statistics
• Query performance and locks
• WAL and checkpoint metrics
• Table and index usage
• Vacuum and analyze statistics

MongoDB

Detection: Process mongod on port 27017 Metrics Collected:

• Database operations
• Collection statistics
• Replication set status
• Sharding metrics
• Memory and storage usage

Redis

Detection: Process redis-server on port 6379 Metrics Collected:

• Memory usage and fragmentation
• Command statistics
• Keyspace information
• Replication status
• Persistence metrics

Message Queues

RabbitMQ

Detection: Process beam.smp with RabbitMQ Metrics Collected:

• Queue depth and processing rates
• Exchange statistics
• Connection and channel metrics
• Memory and disk usage
• Cluster status

Apache Kafka

Detection: Java process with Kafka classes Metrics Collected:

• Topic and partition metrics
• Producer/consumer statistics
• Broker performance
• ZooKeeper coordination
• Log segment metrics

Application Frameworks

Node.js Applications

Detection: Process node with application entry point Discovery Process:

javascriptCopy
// Detected from package.json
{
  "name": "my-api",
  "version": "1.2.3",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  }
}

Metrics Collected:

• Event loop metrics
• Memory heap usage
• HTTP request statistics
• Garbage collection metrics

Java Applications

Detection: Java processes with specific JAR files Discovery Features:

• JMX endpoint detection
• Application server identification (Tomcat, Jetty)
• Framework detection (Spring Boot, Micronaut)
• JVM metrics collection

Python Applications

Detection: Python processes with web frameworks Supported Frameworks:

• Django applications
• Flask applications
• FastAPI services
• Gunicorn/uWSGI servers

Container Platforms

Docker

Detection: Docker daemon and containers Metrics Collected:

• Container resource usage
• Image and volume statistics
• Network metrics
• Runtime information

Container Discovery:

yamlCopy
# Auto-discovered containers
Containers:
  - Name: web-app-1
    Image: nginx:latest
    Status: Running
    Ports: ["80:8080", "443:8443"]
    Labels:
      app: frontend
      version: v1.2.3

Kubernetes

Detection: Kubelet and cluster services Discovery Scope:

• Pod and service discovery
• Deployment and ReplicaSet monitoring
• Namespace organization
• Resource quota tracking

WordPress Monitoring

WordPress Detection

Methods:

• WordPress plugin installation
• File system analysis (wp-config.php)
• Database schema detection
• HTTP header analysis

WordPress-Specific Metrics:

yamlCopy
WordPress Site: example.com
Core Version: 6.3.2
PHP Version: 8.1.12
Database: MySQL 8.0.28
Themes:
  - Active: custom-theme
  - Available: [twentytwentythree, twentytwentytwo]
Plugins:
  - Active: 23 plugins
  - Updates Available: 3 plugins
  - Security Issues: 0 plugins
Performance:
  - Page Load Time: 1.2s average
  - Database Queries: 15 per page
  - Memory Usage: 64MB peak

Configuration Options

Discovery Settings

Enable/Disable Discovery

yamlCopy
# Agent configuration
discovery:
  enabled: true
  scan_interval: 300s  # 5 minutes
  timeout: 30s
  
  # Service-specific discovery
  services:
    web_servers: true
    databases: true
    message_queues: true
    containers: true
    wordpress: true
    
  # Network scanning
  network_scan:
    enabled: true
    port_range: "1-65535"
    exclude_ports: [22, 23, 135, 139, 445]  # Security-sensitive ports

Discovery Filters

yamlCopy
# Include/exclude patterns
discovery:
  include_patterns:
    - "nginx*"
    - "mysql*"
    - "redis*"
    
  exclude_patterns:
    - "*-backup"
    - "test-*"
    - "dev-*"
    
  # Host-specific rules
  host_rules:
    web-servers:
      include: ["apache*", "nginx*"]
      exclude: ["apache-test*"]
    db-servers:
      include: ["mysql*", "postgres*"]
      monitoring_level: "detailed"

Custom Service Definitions

Manual Service Configuration

For services not automatically detected:

yamlCopy
# Custom service definition
custom_services:
  - name: "custom-api"
    type: "http"
    host: "localhost"
    port: 8080
    health_check:
      url: "/health"
      method: "GET"
      expected_status: 200
      timeout: 5s
    metrics:
      - endpoint: "/metrics"
        format: "prometheus"
      - endpoint: "/stats"
        format: "json"
        
  - name: "legacy-database"
    type: "database"
    host: "legacy-db.internal"
    port: 1521
    protocol: "oracle"
    connection:
      username: "monitor_user"
      password_file: "/etc/nodewarden/secrets/db_password"
    queries:
      - name: "active_sessions"
        sql: "SELECT COUNT(*) FROM v$session WHERE status='ACTIVE'"
      - name: "tablespace_usage"
        sql: "SELECT tablespace_name, used_percent FROM dba_tablespace_usage_metrics"

WordPress-Specific Configuration

WordPress Plugin Setup

1. Install Plugin: Download from WordPress repository
2. Configure API Key: Enter Nodewarden credentials
3. Enable Monitoring: Select monitoring features

phpCopy
// WordPress plugin configuration
define('NODEWARDEN_API_KEY', 'your-api-key');
define('NODEWARDEN_CUSTOMER_ID', 'your-customer-id');

// Monitoring options
$nodewarden_config = array(
    'monitor_performance' => true,
    'monitor_security' => true,
    'monitor_updates' => true,
    'monitor_backups' => true,
    'anonymous_data' => false  // Respect privacy
);

WordPress Metrics Collection

yamlCopy
WordPress Monitoring:
  Core Health:
    - WordPress version and updates
    - PHP version and extensions
    - Database version and performance
    - File permissions and security
    
  Performance:
    - Page load times
    - Database query performance
    - Plugin performance impact
    - Cache hit ratios
    
  Security:
    - Plugin vulnerabilities
    - Theme security issues
    - User permissions audit
    - Failed login attempts
    
  Maintenance:
    - Available updates
    - Backup status
    - Disk space usage
    - Error log monitoring

Service Grouping and Tagging

Automatic Tagging

Environment Detection

yamlCopy
# Auto-generated tags
Tags:
  service_type: "web_server"
  application: "nginx"
  version: "1.18.0"
  environment: "production"  # Detected from hostname/config
  role: "frontend"
  team: "web-team"  # From configuration or labels

Application Stack Detection

yamlCopy
# LAMP Stack Detection
Stack: LAMP
Components:
  - linux: "Ubuntu 20.04"
  - apache: "2.4.41"
  - mysql: "8.0.28"
  - php: "8.1.12"
  
Auto-tags:
  - stack:lamp
  - web-tier:frontend
  - db-tier:mysql
  - os:ubuntu

Custom Grouping

Service Groups

yamlCopy
# Logical service grouping
Service_Groups:
  web_tier:
    services: ["nginx", "apache"]
    alerts: "web-team-alerts"
    dashboards: ["web-overview", "web-performance"]
    
  database_tier:
    services: ["mysql", "postgresql", "redis"]
    alerts: "dba-team-alerts"
    dashboards: ["db-overview", "db-performance"]
    
  application_tier:
    services: ["node-app", "python-api", "java-service"]
    alerts: "dev-team-alerts"
    dashboards: ["app-overview", "app-metrics"]

Monitoring Integration

Automatic Dashboard Creation

Service-Specific Dashboards

When a service is discovered, Nodewarden automatically creates:

• Overview Dashboard: Key metrics and health status
• Performance Dashboard: Detailed performance metrics
• Troubleshooting Dashboard: Error rates and diagnostic info

Example: MySQL Auto-Dashboard

yamlCopy
Dashboard: "MySQL - db-server-01"
Widgets:
  - Connection Pool Status
  - Query Performance (QPS, slow queries)
  - InnoDB Metrics (buffer pool, lock waits)
  - Replication Status (if configured)
  - Storage Usage (data, logs, temp)
  - Error Log Summary

Automatic Alert Creation

Default Alert Rules

Based on service type and best practices:

yamlCopy
# MySQL Alert Rules (auto-created)
Alerts:
  - Name: "MySQL Connection Pool Full"
    Condition: connections_used > 90% of max_connections
    Severity: Warning
    
  - Name: "MySQL Slow Queries High"
    Condition: slow_queries_per_second > 10
    Severity: Warning
    
  - Name: "MySQL Replication Lag"
    Condition: replication_lag > 60 seconds
    Severity: Critical
    
  - Name: "MySQL Service Down"
    Condition: service_status != "running"
    Severity: Critical

Troubleshooting Discovery

Common Issues

Services Not Detected

Symptoms: Known services don't appear in Nodewarden Solutions:

1. Check Agent Permissions: Ensure agent can read process information
2. Verify Network Access: Confirm agent can access service ports
3. Review Discovery Logs: Check agent logs for discovery errors
4. Manual Configuration: Add custom service definitions

Incorrect Service Information

Symptoms: Wrong version or configuration detected Solutions:

1. Update Detection Rules: Modify discovery patterns
2. Manual Override: Specify correct information manually
3. Configuration Access: Ensure agent can read config files
4. Process Analysis: Verify process command line parsing

High Discovery Overhead

Symptoms: Discovery consuming too many resources Solutions:

1. Increase Scan Interval: Reduce discovery frequency
2. Limit Scope: Exclude unnecessary services or ports
3. Optimize Patterns: Use more specific include/exclude rules
4. Resource Limits: Set CPU/memory limits for discovery

Discovery Debugging

Debug Commands

bashCopy
# Test service discovery manually
nodewarden-agent discover --verbose

# Show discovered services
nodewarden-agent services list

# Test specific service detection
nodewarden-agent discover --service mysql --debug

# Verify configuration
nodewarden-agent config validate

Discovery Logs

bashCopy
# View discovery logs
sudo journalctl -u nodewarden-agent | grep discovery

# Example log output
INFO[2024-12-05T10:30:00Z] Starting service discovery scan
INFO[2024-12-05T10:30:01Z] Detected MySQL server version 8.0.28 on port 3306
INFO[2024-12-05T10:30:02Z] Detected Nginx server version 1.18.0 on port 80
INFO[2024-12-05T10:30:03Z] WordPress site detected: example.com (version 6.3.2)
INFO[2024-12-05T10:30:05Z] Discovery scan completed: 5 services found

Best Practices

Discovery Optimization

Efficient Scanning

• Targeted Discovery: Focus on relevant services for each host
• Appropriate Intervals: Balance freshness with resource usage
• Network Optimization: Limit port scanning scope
• Permission Management: Provide minimal required access

Security Considerations

• Credential Management: Secure storage of monitoring credentials
• Network Security: Restrict discovery to authorized networks
• Access Control: Limit agent permissions to necessary resources
• Audit Logging: Track discovery activities and access

Service Organization

Consistent Naming

• Standardized Names: Use consistent service naming conventions
• Environment Prefixes: Clearly identify environments (prod-, dev-, test-)
• Team Ownership: Tag services with responsible teams
• Version Tracking: Include version information in tags

Lifecycle Management

• Automated Cleanup: Remove monitoring for decommissioned services
• Migration Tracking: Update monitoring during service migrations
• Deployment Integration: Automatic discovery for new deployments
• Change Management: Track service configuration changes

Next Steps

Now that you understand service discovery:

1. Custom Dashboards - Create views for discovered services
2. Integration Guide - Connect with deployment pipelines
3. Advanced Configuration - Fine-tune discovery settings
4. Automation - Automate service onboarding
5. Security Best Practices - Secure your monitoring setup

Want to customize discovery for your environment? Check out our Advanced Service Discovery Guide for complex scenarios and enterprise features.