Clean up CALM schema project

[rocketstack-matt]

CALM Schema Directory Cleanup

Overview

This issue proposes a cleanup of the calm directory structure to improve organization, remove deprecated examples, and ensure a more maintainable schema repository. The recent introduction of the prototype directories within schema proposals provides a more consistent approach for examples, making several older directories redundant.

Goals

• Simplify the directory structure
• Remove deprecated examples and redundant content
• Create a dedicated project for multiple workshops/demos
• Preserve and migrate valuable TraderX reference architecture
• Ensure proper handling of any dependencies

Proposed Changes

1. Create New Standardized Repositories

Create two new folders at the top level of the calm directory to serve as golden sources for FINOS-published standard components:

• calm/controls/ - Official repository of standardized control definitions

  • Will house validated, approved control requirements
  • Organized by domains (security, performance, compliance, etc.)
  • Versioned appropriately
  • Published to calm-hub as reference controls

• calm/interfaces/ - Official repository of standardized interface definitions

  • Will contain validated, approved interface types
  • Organized by technology domains and communication patterns
  • Versioned appropriately
  • Published to calm-hub as reference interfaces

These repositories will form the foundation for a library of reusable, standardized components that organizations can use directly or extend for their specific needs.

2. Delete Deprecated Example Directories

The following directories contain outdated examples that have been superseded by the prototype directories in draft schema proposals:

• calm/control-example/ - Contains old control examples
• calm/domains-example/ - Contains domain-specific examples
• calm/samples/ - Contains miscellaneous samples, except for the TraderX content

These should be removed as they no longer represent current best practices. Most valuable examples have been migrated to the prototype directories in corresponding schema proposals.

Special Handling for TraderX Content

The TraderX CALM definition currently exists in:

• calm/samples/2024-10/traderx/
• calm/samples/2024-12/traderx/
• calm/draft/2025-03/samples/traderx/

This content represents a valuable reference architecture and should be preserved. Instead of deletion, we should:

1. Enhance the TraderX CALM definition to showcase the latest schema features
2. Migrate the enhanced definition to the TraderX repository
3. Ensure the migrated content is well-documented and accessible
4. Update any references to the old locations

2. Analyze and Address Pattern Directory Dependencies

The calm/pattern/ directory is currently referenced by CLI tests:

cli/src/server/routes/validation-route.spec.ts
cli/src/cli.e2e.spec.ts
cli/README.md

Before removing this directory, we need to:

• Analyze all CLI tests that depend on files in this directory
• Create equivalent test fixtures in the CLI project's test directories
• Update test references to point to the new locations
• Verify all tests pass with the new structure

A separate PR should be created for this change to ensure test integrity is maintained.

3. Create New Workshops Project

The calm/workshop/ directory contains valuable demo materials but doesn't belong in the schema repository. We should:

• Create a new top-level project designed to host multiple workshop materials
• Structure the project to accommodate different workshop types and scenarios
• Move all contents from calm/workshop/ to an appropriate location in this new project
• Allow for easy addition of new workshops in the future
• Ensure documentation is updated to reference the new location
• Update any links or references in other documentation

Implementation Plan

1. Create multi-workshop project and migrate existing content

   • Design and create new project structure to support multiple workshops
   • Set up organization for different workshop types and scenarios
   • Migrate existing workshop materials to appropriate location in new structure
   • Create template for adding future workshops
   • Update documentation
   • Verify functionality in new location

2. Address pattern directory dependencies

   • Create test fixtures in CLI project
   • Update test references
   • Verify all tests pass

3. Enhance and migrate TraderX content

   • Enhance TraderX CALM definition with latest features
   • Set up proper structure in TraderX repository
   • Migrate content and update documentation
   • Create redirects or references as needed

4. Add new controls and interfaces directories and associated GitHub Action to maintain FINOS published controls and interfaces which should be automatically published to the hosted calm-hub (see #1239) when added, deleted or amended.

5. Remove deprecated directories

   • Remove control-example, domains-example, and samples (after TraderX migration)
   • Update documentation
   • Check for any missed references

6. Remove pattern directory

   • Only after all dependencies are addressed
   • Final cleanup and validation

Impact

• Positive: Simplified directory structure, clearer organization, reduced maintenance overhead
• Potential Concerns: Test dependencies need careful handling to avoid breaking changes

Alternative Approaches

1. Keep directories but mark as deprecated: Less disruptive but doesn't fully clean up the repository
2. Move all examples to a separate repository: More complex but could provide better separation of concerns

Additional Notes

• We should ensure proper communication about these changes to anyone who might be using these examples
• Any links to these directories in documentation should be updated
• We should consider creating an archive of these examples if there's concern about losing historical reference