ILogGroup: Subfilter create and delete doesn't work if log group's sub filter has 2 filters

[farazoman]

Describe the bug

When updating a subscription filter via CDK, the diff produces a change which indicated a Subscription Filter is being created and deleted. When running deploy, the sub filter gets created before the old one being deleted. I understand this to be ideal as we don't want loss of data. The problem is that if the log group already has 2 sub filters, then cloudformation will fail because by creating a 3rd subscription filter the provisioning fails everything.

cdk version: "2.176.0"

The error on cloudformation:

Resource handler returned message: "Resource limit exceeded. (Service: CloudWatchLogs, Status Code: 400, Request ID: d2bd233c-01a9-45fd-9e34-1db19eee7f2b) (SDK Attempt Count: 1)" (RequestToken: ae35be86-80e5-682b-dcb6-55ec164912aa, HandlerErrorCode: ServiceLimitExceeded)

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Last Known Working CDK Library Version

No response

Expected Behavior

Provisioning should be successful

Current Behavior

Provisioning fails since there is no space to make a log group

Reproduction Steps

1. provision a log group with 2 subscription filters
2. modify cdk to create and replace one of the two subscription filters
3. see that provisioning fails

Possible Solution

Fix 1: delete first then create -> or an option to allow this depending on the user's circumstance Fix 2: sub filter should be able to allow having 3 sub filters temporarily so that create and replace can happen without data loss

Additional Information/Context

That it should properly provision the filter

AWS CDK Library version (aws-cdk-lib)

├─┬ @myhelix/[email protected] │ ├─┬ @aws-cdk/[email protected] │ │ └── [email protected] deduped │ └── [email protected] deduped ├── [email protected] └─┬ [email protected] └── [email protected] deduped

AWS CDK CLI version

2.176.0 (build 899965d)

Node.js Version

v20.10.0

OS

mac 15.5

Language

TypeScript

Language Version

No response

Other information

No response

[pahud]

Hi @farazoman,

Thank you for reporting this issue. You've identified a legitimate problem with CDK's subscription filter replacement strategy when hitting AWS CloudWatch Logs' 2-filter limit.

Root Cause: CloudFormation creates new resources before deleting old ones to prevent downtime, but this temporarily exceeds the 2 subscription filter limit per log group.

Immediate Workarounds: • Manually delete one existing filter before deployment • Change construct ID to force different logical ID • Use custom resource for replacement logic

Proposed Fix: Enhance the SubscriptionFilter construct to add a replacementStrategy property allowing DELETE_BEFORE_CREATE behavior, plus better validation and error messaging.

Current Workarounds

1. CloudFormation DeletionPolicy Override

const subscriptionFilter = new SubscriptionFilter(this, 'MyFilter', {
  logGroup: myLogGroup,
  destination: newDestination,
  filterPattern: FilterPattern.allEvents()
});


// Force delete-before-create behavior
const cfnFilter = subscriptionFilter.node.defaultChild as CfnSubscriptionFilter;
cfnFilter.addDeletionOverride('DeletionPolicy', 'Delete');
cfnFilter.addPropertyOverride('UpdateReplacePolicy', 'Delete');

Pros: • Directly addresses the root cause • No need to change construct IDs • Maintains existing resource references

Cons: • Complex CloudFormation knowledge required • May not work reliably with all resource types • Hard to predict CloudFormation behavior • Difficult to test and validate

2. Manual Logical ID Management (Recommended)

// When you need to update a filter, change the construct ID
// This forces CloudFormation to treat it as a new resource

// Old deployment:
// new SubscriptionFilter(this, 'MyFilter-v1', { ... });

// New deployment (forces replacement):
new SubscriptionFilter(this, 'MyFilter-v2', {
  logGroup: myLogGroup,
  destination: newDestination,
  filterPattern: FilterPattern.allEvents()
});

// The old filter will be deleted automatically

Pros: • Simple and predictable • Works with CDK's natural resource lifecycle • Easy to implement and understand • Low risk of unintended side effects • Automatic cleanup of old resources • No special CloudFormation knowledge needed

Cons: • Requires manual version tracking • Slightly verbose code • Need to remember to increment version

We encourage community contributions! I think we can introduce a new property like this

export interface SubscriptionFilterProps extends SubscriptionFilterOptions {
  readonly logGroup: ILogGroup;

  /**
   * Strategy for replacing subscription filters when updates occur.
   *
   * When DELETE_BEFORE_CREATE is used, the old subscription filter will be deleted
   * before creating the new one. This prevents exceeding AWS CloudWatch Logs'
   * limit of 2 subscription filters per log group but may cause brief data loss.
   *
   * @default ReplacementStrategy.CREATE_BEFORE_DELETE
   */
  readonly replacementStrategy?: ReplacementStrategy;
}

export enum ReplacementStrategy {
  /**
   * Create new resource before deleting old one (default CloudFormation behavior)
   * Ensures no data loss but may hit resource limits
   */
  CREATE_BEFORE_DELETE = 'CreateBeforeDelete',

  /**
   * Delete old resource before creating new one
   * Prevents resource limit issues but may cause brief data loss
   */
  DELETE_BEFORE_CREATE = 'DeleteBeforeCreate',
}

I am making it a p2 and we welcome community PRs!

[Kasra-G]

This is a cloudformation limitation, no? I don't think cloudformation supports a "delete before create" system. UpdateReplace policy just dictates what happens to the old resource once it is replaced by the new resource (do we retain or delete it). If the number of subscriptions per log group is a service quota limit then you may be able to request a quota increase, otherwise you will need to do the 2 step deployment as you outlined.

https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-attribute-updatereplacepolicy.html

For example, if you update the Engine property of an AWS::RDS::DBInstance resource type, CloudFormation creates a new resource and replaces the current DB instance resource with the new one. The UpdateReplacePolicy attribute would then dictate whether CloudFormation deleted, retained, or created a snapshot of the old DB instance.

Another option is to have only one subscription filter that streams everything to an SNS topic (perhaps with a Lambda). Then you can use SNS topic subscription filters to perform the filtering you need.

[pahud]

@Kasra-G

Yes you are right. My previous triage comment was flawed.

Thank you @HsiehShuJeng for this well-implemented PR and the comprehensive testing! I also want to acknowledge @Kasra-G's excellent technical analysis in the original issue.

After further review and consideration of @Kasra-G's feedback, I need to revise my initial assessment. Kasra-G is absolutely correct - this is fundamentally a CloudFormation limitation that cannot be solved with UpdateReplacePolicy and DeletionPolicy attributes.

I recommend users adopt @Kasra-G's architectural approach:

// Single subscription filter that handles all log streaming
const mainSubscriptionFilter = new logs.SubscriptionFilter(this, 'MainFilter', {
  logGroup: myLogGroup,
  destination: new destinations.SnsDestination(topic), // or Lambda
  filterPattern: logs.FilterPattern.allEvents(), // Stream everything
});

// Use SNS topic subscriptions for filtering
new sns.Subscription(this, 'ErrorSubscription', {
  topic: topic,
  endpoint: errorHandlerLambda.functionArn,
  protocol: sns.SubscriptionProtocol.LAMBDA,
  filterPolicy: {
    level: sns.SubscriptionFilter.stringFilter({
      allowlist: ['ERROR', 'FATAL']
    })
  }
});

new sns.Subscription(this, 'InfoSubscription', {
  topic: topic,  
  endpoint: infoHandlerLambda.functionArn,
  protocol: sns.SubscriptionProtocol.LAMBDA,
  filterPolicy: {
    level: sns.SubscriptionFilter.stringFilter({
      allowlist: ['INFO', 'DEBUG']
    })
  }
});

This approach:

• Avoids the 2-filter limit entirely
• Provides more flexible filtering options
• Scales better (unlimited SNS subscriptions)
• Uses standard AWS services without workarounds

I'm going to close this PR and update the original issue to:

• nDocument the CloudFormation limitation
• Provide the SNS-based architectural pattern as the recommended solution
• Remove the misleading workaround suggestions from my original comment

Thank you again for the excellent implementation work - the code quality and testing were top-notch! The issue was with the approach, not the execution.

cc @Kasra-G - thank you for the sharp technical insight that helped us avoid shipping a non-functional feature.

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.