apiops icon indicating copy to clipboard operation
apiops copied to clipboard

Add comprehensive YAML configuration validation system for extractor

Open MO2k4 opened this issue 3 months ago โ€ข 1 comments

๐ŸŽฏ Problem Solved

Currently, the APIops extractor tool processes configuration.extractor.yaml files without validation, leading to runtime errors that are difficult to diagnose when configuration files contain:

  • Empty sections (e.g., empty API lists)
  • Duplicate entries across different sections
  • Invalid data types or malformed YAML structure
  • Names with invalid characters or whitespace

Users often encounter cryptic error messages during extraction, making it challenging to identify and fix configuration issues quickly.

๐Ÿ› ๏ธ Solution Implemented

This PR introduces a comprehensive YAML configuration validation system that:

Core Features:

  • ๐Ÿ” Pre-runtime validation - Catches configuration errors before extraction begins
  • ๐ŸŽฏ Detailed error reporting - Provides clear error messages with YAML line numbers
  • โšก Multiple validation modes - Automatic validation during runtime + standalone CLI tool
  • ๐Ÿ”ง Comprehensive rules - Validates structure, content, types, and naming conventions

Validation Rules:

  1. Empty Section Detection - Warns about empty APIs, Products, Groups, etc.
  2. Duplicate Entry Prevention - Ensures no duplicate names across all configuration sections
  3. Type Validation - Confirms all entries are strings (not objects/arrays)
  4. Naming Convention Enforcement - Prevents whitespace and special characters in resource names

Integration Points:

  • Automatic validation during extractor startup (fails fast on invalid config)
  • Standalone CLI tool: ./extractor validate-config <file> for manual validation
  • Consistent error handling using existing LanguageExt Either<T,U> patterns

๐Ÿ“ Files Added/Modified

New Files:

  • tools/code/common/ConfigurationValidator.cs - Core validation logic
  • tools/code/extractor/ConfigurationValidationCommand.cs - CLI validation tool
  • tools/code/extractor-config/configuration.extractor.example.yaml - Valid example
  • tools/code/extractor-config/configuration.extractor.invalid-example.yaml - Invalid example for testing

Modified Files:

  • tools/code/common/Configuration.cs - Integrated validation into config loading
  • tools/code/extractor/Configuration.cs - Added extractor-specific validation
  • tools/code/extractor/App.cs - Added validation at application startup
  • tools/code/extractor/Program.cs - Added validation command support
  • docs/apiops/3-apimTools/apiops-2-1-tools-extractor.md - Updated documentation
  • tools/README.md - Added validation guidelines
  • configuration.extractor.yaml - Added validation comments

๐Ÿ“‹ Usage Examples

Automatic Validation (during extraction):

./extractor extract --configuration-file invalid-config.yaml
# Output: Configuration validation failed: Duplicate entry 'my-api' found in both apis and products sections (line 15)

Manual Validation:

./extractor validate-config ./configuration.extractor.yaml
# Output: โœ… Configuration validation passed successfully!

./extractor validate-config ./invalid-config.yaml
# Output: โŒ Configuration validation failed:
#         - Empty section detected: 'groups' section is empty (line 25)
#         - Duplicate entry 'test-api' found in both apis and products sections (line 30)

Error Message Examples:

  • Empty section detected: 'apis' section is empty (line 10)
  • Duplicate entry 'my-product' found in both products and groups sections (line 25)
  • Invalid entry type: Expected string but found object for entry 'test-api' (line 15)
  • Invalid naming convention: Entry 'my api' contains whitespace or special characters (line 20)

๐Ÿงช Testing

  • โœ… Manual testing with valid and invalid configuration files
  • โœ… Integration testing with existing extractor workflow
  • โœ… Error handling verification with various malformed YAML scenarios
  • โœ… CLI tool testing for standalone validation scenarios

๐Ÿ“– Documentation

  • Updated extractor documentation with validation guidelines
  • Added example configuration files with proper structure
  • Included troubleshooting section for common validation errors
  • Enhanced tools README with validation workflow

๐Ÿ”„ Backward Compatibility

  • โœ… Fully backward compatible - No breaking changes to existing functionality
  • โœ… Graceful degradation - Validation failures provide clear guidance for fixes
  • โœ… Optional CLI tool - Manual validation is opt-in, doesn't affect existing workflows

๐ŸŽ Benefits for Users

  1. ๐Ÿš€ Faster debugging - Immediate feedback on configuration issues
  2. ๐Ÿ“ Better error messages - Clear, actionable error descriptions with line numbers
  3. ๐Ÿ” Proactive validation - Catch issues before they cause extraction failures
  4. ๐Ÿ› ๏ธ Developer experience - Standalone validation tool for configuration development
  5. ๐Ÿ“š Better documentation - Clear examples and validation guidelines

MO2k4 avatar Sep 25 '25 06:09 MO2k4

@MO2k4 - thanks for the PR. I unfortunately have to reject it as-is. We made a lot of configuration changes in our next version, which will conflict with your updates.

You can take a look at the v7 code to see where we're going. The CHANGELOG is incomplete but shows some configuration changes we made. Happy to get feedback.

guythetechie avatar Sep 25 '25 14:09 guythetechie