ctlstore
ctlstore copied to clipboard
segmentio
Control Data Store
1 . Snyk has detected vulnerabilities in the docker image(s) golang:alpine golang:1.15 used in this repository. Please refer to the [docker security metrics](https://app.snowflake.com/us-west-2/segmentdataeng/w1iVJjEKKQ7j) to know the exact number and severity...
# Description [CF-557](https://segment.atlassian.net/browse/CF-557) Ensure that we don't propagate changelog updates until after a transaction is committed. Prior to this fix, every DML statement processed (even within a transaction) would immediately...
> **Note** > This is a resurrection of Collin Van Dyck's attempt to do this: > * https://github.com/segmentio/ctlstore/tree/go-sqlite > > See Colin King's version here: https://github.com/segmentio/ctlstore/pull/128 --- Better benchmarks Break...
> **Note** > This is a resurrection of Colin King's attempt to do this: > * https://github.com/segmentio/ctlstore/tree/colin-zoomify-sqlite > > See Collin Van Dyck's version here: https://github.com/segmentio/ctlstore/pull/129 --- * switch from...
I have opened this PR for experimental purposes to resolve the OOM issue in warehouses. Once I get approval for my approach, I will add other things such as test...
``` WARNING: DATA RACE -- | Write at 0x000000e93ac0 by goroutine 22: | database/sql.convertAssignRows() | /go/src/database/sql/convert.go:381 +0x1aa7 | database/sql.(*Rows).Scan() | /go/src/database/sql/sql.go:3326 +0x6e4 | github.com/segmentio/ctlstore/pkg/scanfunc.New.scanFuncStruct.func1() | /var/lib/buildkite-agent/go/pkg/mod/github.com/segmentio/[email protected]/pkg/scanfunc/scan_func.go:88 +0xa7 | github.com/segmentio/ctlstore.(*Rows).Scan() |...
- Replace `aws-sdk-go` with `aws-sdk-go-v2` - Use concurrent downloading mechanism to download s3 snapshot with hardcoded concurrency configuration - replace golang's `gzip` with `klauspost/gizp`
Check out https://twilio.slack.com/archives/C0324FBR4Q5/p1677021062766279 for RCA. In order to make this change effective to every node loading ldb, we might need to cut a tag and making every terraform places updating...