rspec
Make build script more readable
Hello
As mentioned in this comment : https://github.com/rspec/rspec-rails/pull/2231#issuecomment-567930499
I would love to be able to more easily run the build locally, and run each step without looking at shell functions or Rakefile. Like simply copy paste one line from travis logs.
I have often problems to replay a failing step especially on rspec-rails. Most of the time I open the Travis build's log, try to rerun the step, if I am not able to reproduce the error, I rerun previous or parent task, then if I'm still not able to reproduce the error, I try to be as close as possible to the CI environment.
- Does CI build steps are clear enough?
I think we should improve the command that are executed. Maybe with some visual space, ascii separator. We should be able to easily see all commands that are executed by the CI with the relative path. I am thinking about tasks in "example app" in rspec-rails that does not mention the path where they are executed.
- How to more easily reproduce the step?
Should we provide when the command fail, the full command to rerun locally the step? For example:
🚧 Build failed at step "bundle exec rake smoke:app". To rerun the step:
rvm use 2.6 && rm -f Gemfile.lock && export RAILS_VERSION='~>6.0' && bundle --binstubs && bundle exec rake smoke:app
Maybe a bash script that would choose ruby via rvm setup, check binstubs then run a specific commands like above. Something like:
🚧 Build failed at step "bundle exec rake smoke:app". To rerun the step:
script/local_retry --ruby_version='2.6' --command='bundle exec rake smoke:app'`
I looked at Github actions, to see if it was more readable. It is a little bit better to display step, but not executed commands. https://github.com/dry-rb/dry-effects/runs/358219156
So the problem is not our CI host, the problem is our build consists of various chunks which are a combination of bash scripts and rake tasks which generate apps and run tasks that are not easily replicated. I think focusing on making those more replicatable will help.
I think the way rspec-rails 4 goes is the way that will help us for other rspec gems. Having a smaller build matrix is a game changer.
I don't see at the moment the gain of using Github action instead of Travis CI.