Lingvo builds unexpected target

[oplatek]

Hi,

I have prepared a custom local build where the following command passes

bazel test -c opt --test_output=streamed //lingvo:trainer_test //lingvo:models_test 

However, when I run

$ bazel build -c opt //lingvo:trainer
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
DEBUG: Rule 'subpar' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "35bb9f0092f71ea56b742a520602da9b3638a24f", shallow_since = "1557863961 -0400" and dropping ["tag"]
DEBUG: Call stack for the definition of repository 'subpar' which is a git_repository (rule definition at /workspace/oplatek/.cache/bazel/_bazel_oplatek/19dd353501aebeb03a7543d3534ff2b4/external/bazel_tools/tools/build_defs/repo/git.bzl:181:18):
 - /workspace/oplatek/code/lingvo/WORKSPACE:11:1
INFO: Analyzed target //lingvo:trainer (45 packages loaded, 3569 targets configured).
INFO: Found 1 target...
Target //lingvo:trainer up-to-date:
  bazel-out/k8-py2-opt/bin/lingvo/trainer
INFO: Elapsed time: 84.510s, Critical Path: 21.56s
INFO: 36 processes: 36 linux-sandbox.
INFO: Build completed successfully, 51 total actions

I expect bazel-bin/lingvo/trainer target to be created instead of bazel-out/k8-py2-opt/bin/lingvo/trainer

Any advice on what to do differently to get bazel-bin/lingvo/trainer?

Thank you for your help

[oplatek]

I was able to reproduce it with using following snippet and the docker/dev.dockerfile. It is a property of the commit f8dbbd8bdc80bcf47430f692097974de32877b1d

#!/bin/bash
set -e
set -o pipefail
set -x

LINGVO_DIR="$(pwd)"  # (change to the cloned lingvo directory, e.g. "$HOME/lingvo")
LINGVO_DEVICE=""  # (Leave empty to build and run CPU only docker)
docker build --tag tensorflow:lingvo $(test "$LINGVO_DEVICE" = "gpu" && echo "--build-arg base_image=nvidia/cuda:10.0-cudnn7-runtime-ubuntu16.04") - < ${LINGVO_DIR}/docker/dev.dockerfile

docker run --rm $(test "$LINGVO_DEVICE" = "gpu" && echo "--runtime=nvidia") -t -v ${LINGVO_DIR}:/tmp/lingvo -v ${HOME}/.gitconfig:/home/${USER}/.gitconfig:ro -p 6006:6006 -p 8888:8888 --name lingvo-test-docker-batch tensorflow:lingvo bazel test -c opt --test_output=streamed //lingvo:trainer_test //lingvo:models_test

docker run --rm $(test "$LINGVO_DEVICE" = "gpu" && echo "--runtime=nvidia") -t -v ${LINGVO_DIR}:/tmp/lingvo -v ${HOME}/.gitconfig:/home/${USER}/.gitconfig:ro -p 6006:6006 -p 8888:8888 --name lingvo-test-docker-train tensorflow:lingvo bash -c "bazel build -c opt //lingvo:trainer && [ -f ./bazel-out/k8-py2-opt/bin/lingvo/trainer ] && mkdir -p /tmp/mnist && bazel run -c opt //lingvo/tools:keras2ckpt -- --dataset=mnist --out=/tmp/mnist/mnist && ./bazel-out/k8-py2-opt/bin/lingvo/trainer --run_locally=cpu --mode=sync --model=image.mnist.LeNet5 --logdir=/tmp/mnist/log --logtostderr"

[oplatek]

Note, that the "built binary"/script ./bazel-out/k8-py2-opt/bin/lingvo/trainer works fine when I use the dev.dockerfile

[drpngx]

It seems that bazel changed the output. Feel free to send a PR to update the README and run_distributed.

On Tue, Sep 17, 2019 at 11:41 AM Ondrej Platek [email protected] wrote:

Note, that the "built" binary ./bazel-out/k8-py2-opt/bin/lingvo/trainer works fine when I use the dev.dockerfile

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/lingvo/issues/154?email_source=notifications&email_token=AE75E3ILYBQFFTFG3TR7PFTQKEQENA5CNFSM4IXRZ4WKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65QBAQ#issuecomment-532349058, or mute the thread https://github.com/notifications/unsubscribe-auth/AE75E3NLDWWMMEN2AFR7NALQKEQENANCNFSM4IXRZ4WA .

[jonathanasdf]

This is known due to moving to python3 support in bazel. We'll wait a few weeks then change everything to python3 by default and that should fix things.