`cargo-chef` doesn't cache local dependencies

[prestontw]

(The repo this issue is based on is private. I will describe the issue first, then try to recreate in a public minimal repo for further clarity and reproducibility.)

I have a project directory with structure

Cargo.toml  # [virtual manifest](https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-manifest) cargo.toml
src
  - libA
  - libB
  - libC
  - serviceZ
  - serviceY
  - serviceX
  - ...
...

Each of the services has individual Cargo.toml files and its own Dockerfile.

I've tried adding cargo-chef to one of these services (currently, we are either not being cache friendly or using a cargo new approach), say, serviceZ. cargo-chef does excellently at caching external dependencies for the entire workspace, but because it skelefies libA, libB, and libC, it has to rebuild these local library dependencies for each change in serviceZ. (The actual repo is too large to individually specify what dependencies to copy over and doing so would lead to keeping dependencies between the project and the Cargo.toml file in sync.) These end up consuming most of the compilation time: using cargo chef can bring subsequent runs from 404s down to 270s, but using a cargo new approach brings subsequent runs down from 404s to 70s. I can also visually see that we are recompiling local dependencies when running cargo build in the cargo chef approach while only serviceZ is compiled in the cargo new approach.

This is related to https://github.com/LukeMathWalker/cargo-chef/issues/74 and https://github.com/LukeMathWalker/cargo-chef/issues/64 in that I want very fine control over what I skelefy.

A quick aside: the cargo new approach suffers from changes in serviceZ invalidating the entire COPY src src line, leading to discarding the dependency cache on source changes. The way I've gotten around it is to separate this process into several stages, similar to the approach in cargo chef, but instead of sharing the recipe.json file between stages, I'm sharing an entire directory that should be the same between runs:

FROM rust as assembler
WORKDIR /usr/app
COPY src src
RUN rm -rf src/serviceZ
RUN USER=root cargo new --bin ./src/serviceZ
COPY src/serviceZ/Cargo.toml src/serviceZ/
# because the above stage should be the same between changes to `serviceZ`, 
# we can cache copying it to other stages

FROM rust as cacher
WORKDIR /usr/app
COPY Cargo.lock Cargo.lock
COPY rust-patches rust-patches
COPY vendored vendored
COPY .cargo .cargo
COPY Cargo.toml Cargo.toml
COPY --from=assembler /usr/app/src ./src
RUN cargo build --offline -p serviceZ && rm target/debug/deps/serviceZ*

FROM rust as builder
# actually copies over contents of serviceZ and builds it using cached items in cacher stage
...