EntityFrameworkCore.ClickHouse icon indicating copy to clipboard operation
EntityFrameworkCore.ClickHouse copied to clipboard

Concurrency issue when deleting

Open werebear73 opened this issue 9 months ago • 0 comments

Using these example files to duplicate my issue.

Program.cs

using ClickHouseDB;
using Microsoft.EntityFrameworkCore;

// See https://aka.ms/new-console-template for more information
Console.WriteLine("Hello, World!");

await using var context = new ClickhouseDbContext();
await context.Database.EnsureCreatedAsync();

ReBuildPricingTable();
FillPricingTable();

var rates = context.Prices;
Console.WriteLine($"Before Number of prices: {rates.Count()}");

Console.WriteLine("\n------------------------------------\nRate Before Changes");
foreach (var i in rates)
    Console.WriteLine($"{i.code} ({i.name}) = {i.rate} at 0/{i.initial_increment}/{i.subsequential_increment}");

List<PricingTableElement> codesToDelete = rates.Where(x => x.code == "44").ToList<PricingTableElement>();
codesToDelete.Add(rates.Where(x => x.code == "4400").ToList<PricingTableElement>()[0]);
//codesToDelete.ForEach(item => item.version++);


rates.RemoveRange(codesToDelete);
Console.WriteLine("\n------------------------------------\nRate After Changes Before Save");
foreach (var i in rates)
    Console.WriteLine($"{i.code} ({i.name}) = {i.rate} at 0/{i.initial_increment}/{i.subsequential_increment}");
context.SaveChanges();

Console.WriteLine("\n------------------------------------\nRate After Changes After Save");
foreach (var i in rates)
    Console.WriteLine($"{i.code} ({i.name}) = {i.rate} at 0/{i.initial_increment}/{i.subsequential_increment}");

Console.WriteLine($"After Number of prices: {rates.Count()}");

void ReBuildPricingTable(){
    // Delete Table
    context.Database.ExecuteSqlRaw("DROP TABLE IF EXISTS x_pricing");
    // Create Table
    context.Database.ExecuteSqlRaw("""
        CREATE TABLE IF NOT EXISTS x_pricing (
        `code` String,
        `name` String,
        `rate` Decimal(10, 6),
        `initial_increment` Int32,
        `subsequential_increment` Int32,
        `version` UInt64,
        `is_deleted` boolean DEFAULT false
    ) ENGINE = ReplacingMergeTree(version, is_deleted) ORDER BY code
    """);
}

void FillPricingTable(){
    context.Database.ExecuteSqlRaw("""
        INSERT INTO x_pricing(code, name, rate, initial_increment,subsequential_increment)
        SELECT '1','United States',0.01,1,1
        UNION ALL
        SELECT '44','United Kingdom',0.03,1,1
        UNION ALL
        SELECT '4400','United Kingdom - Special',0.30,1,1
    """);
}

ClickHouseDB.cs

using ClickHouse.EntityFrameworkCore;
using ClickHouse.EntityFrameworkCore.Extensions;
using Microsoft.EntityFrameworkCore;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Reflection.Metadata;

namespace ClickHouseDB;
class PricingTableElement
{
    public string code { get; set; }

    public string name { get; set; }


    public decimal rate { get; set; }

    public int initial_increment { get; set; }
    public int subsequential_increment { get; set; }

    public UInt64 version {get; set;} 
    public bool is_deleted { get; set; } = false;

}

class ClickhouseDbContext : DbContext
{
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        base.OnConfiguring(optionsBuilder);
        optionsBuilder.UseClickHouse("Host=localhost;Protocol=http;Port=18123;Database=default;Username=default;Password="
        , options =>{ })
            ;
    }
    public DbSet<PricingTableElement> Prices { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        var entityTypeBuilder = modelBuilder.Entity<PricingTableElement>().HasQueryFilter(x => x.is_deleted ==false);
        
        entityTypeBuilder.HasKey(e => e.code);
        entityTypeBuilder.Property(e => e.code).HasColumnName("code");
        entityTypeBuilder.Property(e => e.name).HasColumnName("name");
        entityTypeBuilder.Property(e => e.rate).HasColumnName("rate");
        entityTypeBuilder.Property(e => e.initial_increment).HasColumnName("initial_increment");
        entityTypeBuilder.Property(e => e.subsequential_increment).HasColumnName("subsequential_increment");
        entityTypeBuilder.Property(e => e.version).ValueGeneratedOnAddOrUpdate().IsConcurrencyToken(); //.IsRowVersion()
        entityTypeBuilder.Property(e => e.is_deleted).HasColumnName("is_deleted").HasDefaultValue(false).ValueGeneratedOnUpdateSometimes();
        
        entityTypeBuilder.ToTable("x_pricing", table => table
            .HasReplacingMergeTreeEngine("version", "is_deleted")
            .WithPrimaryKey("code"));
        
    }
}

When I try to delete a range of elements in a list, I get a DB Exception

Unhandled exception. Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See https://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions. at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ThrowAggregateUpdateConcurrencyException(RelationalDataReader reader, Int32 commandIndex, Int32 expectedRowsAffected, Int32 rowsAffected) at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ConsumeResultSetWithRowsAffectedOnly(Int32 commandIndex, RelationalDataReader reader) at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.Consume(RelationalDataReader reader) at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection connection) at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable1 commandBatches, IRelationalConnection connection) at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList1 entries) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList1 entriesToSave) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(StateManager stateManager, Boolean acceptAllChangesOnSuccess) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.<>c.<SaveChanges>b__112_0(DbContext _, ValueTuple2 t) at Microsoft.EntityFrameworkCore.Storage.NonRetryingExecutionStrategy.Execute[TState,TResult](TState state, Func3 operation, Func3 verifySucceeded) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess) at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess) at Microsoft.EntityFrameworkCore.DbContext.SaveChanges() at Program.<Main>$(String[] args) in C:\Users\sam_w\source\repos\clickhouse-net-core-tests\Program.cs:line 29 at Program.<Main>$(String[] args) in C:\Users\sam_w\source\repos\clickhouse-net-core-tests\Program.cs:line 35 at Program.<Main>(String[] args)

It does delete the first element in the list but not the others. I have applied the changes described in https://go.microsoft.com/fwlink/?LinkId=527962. But it has the same behavior except with some modifications it will delete the first then throw the error and loop through deleting the first record and erroring again.

werebear73 avatar Apr 23 '25 22:04 werebear73