leeway icon indicating copy to clipboard operation
leeway copied to clipboard

feat: auto-convert Go library deps to weak dependencies

Open iQQBot opened this issue 2 weeks ago • 0 comments

Summary

Go packages with packaging: library are now automatically treated as "weak dependencies" when referenced, improving build parallelism for Go monorepos.

Motivation

In Go monorepos, libraries are typically used for their source code via go.mod replace directives, not for their built artifacts. Previously, leeway would:

  1. Build the library first (blocking)
  2. Extract the built artifact
  3. Then build the dependent package

This was inefficient because the dependent package only needs the library's source files, not its built artifact.

Changes

Weak Dependencies for Go Libraries

When a Go package with packaging: library is listed as a dependency, it's automatically converted to a "weak dependency":

Aspect Regular Dependency Go Library (Weak)
Affects package version
Must be built first
What's copied to _deps/ Built artifact Source files
go.mod replace added
Build runs Sequentially In parallel

Important Constraint

A Go library can only be a weak dependency if all its transitive dependencies are also Go libraries. If a Go library depends on non-Go-library packages (e.g., generic packages), it's treated as a regular hard dependency.

# CAN be weak dep - only depends on other Go libraries
go-lib-a:
  type: go
  config:
    packaging: library
  deps:
    - other-go-lib:lib

# MUST be hard dep - depends on a generic package  
go-lib-b:
  type: go
  config:
    packaging: library
  deps:
    - some-generic:pkg

Build Flow

Before:
1. Build lib-a (blocking)
2. Build lib-b (blocking) 
3. Build app

After (when libs have only Go library deps):
1. Start in parallel:
   ├── Build app (copies lib-a, lib-b sources to _deps/)
   ├── Build lib-a (runs tests)
   └── Build lib-b (runs tests)

Weak Dependency Failure Propagation

If a weak dependency fails (e.g., tests fail), all packages that depend on it will also fail before running their build commands. This prevents publishing packages (e.g., Docker images) when their weak dependencies have failed.

Example:
  docker:img -> go-app:app -> go-lib:lib (weak dep)
  
If go-lib tests fail:
1. go-lib:lib build fails
2. go-app:app waits for go-lib, sees failure, fails before building
3. docker:img waits for go-lib (via go-app), sees failure, fails before docker push

Implementation uses a broadcast channel pattern - multiple packages can wait on the same weak dep result.

Cache Behavior

  • Source files are always copied from the workspace (not from cache)
  • Version manifest includes all transitive weak deps
  • Any change to a library invalidates the dependent package's cache

Example

# my-lib/BUILD.yaml
packages:
- name: lib
  type: go
  config:
    packaging: library  # Automatically becomes weak dep when referenced

# my-app/BUILD.yaml  
packages:
- name: app
  type: go
  deps:
    - my-lib:lib  # Sources copied, builds in parallel
  config:
    packaging: app

Testing

  • Added unit tests for canBeWeakDep() and checkAllDepsAreGoLibraries()
  • Added unit tests for GetTransitiveWeakDependencies()
  • Added unit tests for collectWeakDependencies()
  • Added tests for auto-conversion behavior
  • Added tests for version manifest inclusion
  • All existing tests pass
  • All integration tests pass

iQQBot avatar Jan 14 '26 19:01 iQQBot