gobuffalo
Adding SQLite support after the fact needs better documentation and doesn't work properly
Description
Steps to Reproduce the Problem
-
$ go get -u -v -tags sqlite github.com/gobuffalo/buffalo/buffalo -
$ ~/go/bin/buffalo new test && cd test - `$ ~/go/bin/buffalo pop create -a v4.10.0
[POP] 2019/04/15 16:05:58 warn - unable to load connection development: could not create new connection: sqlite3 support was not compiled into the binary`
Expected Behavior
[POP] 2019/04/15 16:05:58 info - hey cool things worked and this trivial problem was tested in CI
Actual Behavior
Sqlite3 process per the documentation does not result in sqlite3 support
Info
-> Go: Checking installation
✓ The `go` executable was found on your system at: /usr/local/go/bin/go
-> Go: Checking minimum version requirements
✓ Your version of Go, 1.12.3, meets the minimum requirements.
-> Go: Checking GOPATH
✓ You appear to be operating inside of your GOPATH.
-> Go: Checking Package Management
⚠ You do not appear to be using a package management system.
It is strongly suggested that you use one of the following package management systems:
* Go Modules (Recommended) - https://gobuffalo.io/en/docs/gomods
* Dep - https://github.com/golang/dep
For help setting up your Go environment please follow the instructions for you platform at:
https://www.gopherguides.com/courses/preparing-your-environment-for-go-development
-> Go: Checking PATH
✘ Your PATH (/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:~/Library/Python/2.7/bin) does not contain /Users/<CUT>/go/bin.
Without /Users/<CUT>/go/bin in your `PATH` any Go executables can not be run globally.
For help setting up your Go environment please follow the instructions for you platform at:
https://www.gopherguides.com/courses/preparing-your-environment-for-go-development
-> Node: Checking installation
✓ The `node` executable was found on your system at: /usr/local/bin/node
-> Node: Checking minimum version requirements
✓ Your version of Node, v10.15.3, meets the minimum requirements.
-> NPM: Checking installation
✓ The `npm` executable was found on your system at: /usr/local/bin/npm
-> NPM: Checking minimum version requirements
✓ Your version of NPM, 6.4.1, meets the minimum requirements.
-> Yarn: Checking installation
✓ The `yarnpkg` executable was found on your system at: /usr/local/bin/yarnpkg
-> Yarn: Checking minimum version requirements
✓ Your version of Yarn, 1.15.2, meets the minimum requirements.
-> PostgreSQL: Checking installation
✘ The `postgres` executable could not be found on your system.
For help setting up your Postgres environment please follow the instructions for you platform at:
https://www.postgresql.org/download/
-> MySQL: Checking installation
✘ The `mysql` executable could not be found on your system.
For help setting up your MySQL environment please follow the instructions for you platform at:
https://www.mysql.com/downloads/
-> SQLite3: Checking installation
✓ The `sqlite3` executable was found on your system at: /usr/bin/sqlite3
-> SQLite3: Checking minimum version requirements
✓ Your version of SQLite3, 3.24.0, meets the minimum requirements.
-> Cockroach: Checking installation
✘ The `cockroach` executable could not be found on your system.
For help setting up your Cockroach environment please follow the instructions for you platform at:
https://www.cockroachlabs.com/docs/stable/
-> Buffalo: Checking installation
✘ The `buffalo` executable could not be found on your system.
For help setting up your Buffalo environment please follow the instructions for you platform at:
https://gobuffalo.io/en/docs/installation
For help setting up your Go environment please follow the instructions for you platform at:
https://www.gopherguides.com/courses/preparing-your-environment-for-go-development
-> Buffalo: Application Details
Pwd /Users/<SNIP>/go/src/<SNIP>
Root /Users/<SNIP>/go/src/<SNIP>
GoPath /Users/<SNIP>/go
PackagePkg <SNIP>
ActionsPkg <SNIP>/actions
ModelsPkg <SNIP><SNIP>/models
GriftsPkg <SNIP>/grifts
WithModules false
Name gamoto
Bin bin/gamoto
VCS git
WithPop true
WithSQLite false
WithDep false
WithWebpack true
WithNodeJs true
WithYarn true
WithDocker true
WithGrifts true
AsWeb true
AsAPI false
InApp true
PackageJSON {map[]}
-> Buffalo: config/buffalo-app.toml
name = "gamoto"
bin = "bin/gamoto"
vcs = "git"
with_pop = true
with_sqlite = false
with_dep = false
with_webpack = true
with_nodejs = true
with_yarn = true
with_docker = true
with_grifts = true
as_web = true
as_api = false
-> Buffalo: config/buffalo-plugins.toml
[[plugin]]
binary = "buffalo-plugins"
go_get = "github.com/gobuffalo/buffalo-plugins"
[[plugin]]
binary = "buffalo-pop"
go_get = "github.com/gobuffalo/buffalo-pop"
-> Go: Checking PATH ✘ Your PATH (/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:~/Library/Python/2.7/bin) does not contain /Users/<CUT>/go/bin. Without /Users/<CUT>/go/bin in your `PATH` any Go executables can not be run globally. For help setting up your Go environment please follow the instructions for you platform at: https://www.gopherguides.com/courses/preparing-your-environment-for-go-development -> Buffalo: Checking installation ✘ The `buffalo` executable could not be found on your system. For help setting up your Buffalo environment please follow the instructions for you platform at: https://gobuffalo.io/en/docs/installation For help setting up your Go environment please follow the instructions for you platform at: https://www.gopherguides.com/courses/preparing-your-environment-for-go-development
This could be an issue with you path setup. Can you check that buffalo-plugins and buffalo-pop are in your path and try again?
Buffalo is using plugins for different things, pop is one of those.
Please head to https://gobuffalo.io/en/docs/plugins for more information on this.
In theory, it should work without them being in $PATH, as long as $BUFFALO_PLUGIN_PATH or $GO_PATH are set.
As I could not reproduce this issue on my machine, I can't solve your problem without your help.
If a path setup causes this issue that’s a whole bug of its own
Well, this is a bug report after all, isn't it?
Also, it's specified as a buffalo requirement:
Before installing make sure you have the required dependencies installed:
- A working Go environment
- A configured $PATH environment variable that includes $GOPATH/bin.
- Go version 1.10.8 or greater.
I can't reproduce either, but I believe I know what happened.
You reported your second step as $ ~/go/bin/buffalo new test && cd test and then, correct me if I'm wrong, you edited the database.yml to change it to sqlite, correct? Nothing wrong with doing that, and with any other db you wouldn't have had trouble, but sqlite is a bit trickier.
The problem stems from the app not being generated initially with --db-type=sqlite3. Had you done that, the plugins system would've installed buffalo-pop for you with sqlite support baked in. Your config/buffalo-plugins.toml file would have looked like this:
[[plugin]]
binary = "buffalo-plugins"
go_get = "github.com/gobuffalo/buffalo-plugins"
tags = ["sqlite"]
[[plugin]]
binary = "buffalo-pop"
go_get = "github.com/gobuffalo/buffalo-pop"
tags = ["sqlite"]
The fix would be to either edit your config/buffalo-plugins.toml to include the sqlite tags.
Or you can add it with the buffalo plugins command:
$ buffalo plugins add --tags sqlite github.com/gobuffalo/buffalo-pop
Either should solve your problem. I think the real bug here is some missing documentation. Thanks for letting us know.
@lukasschlueter I'm transferring this to the gobuffalo/gobuffalo repo so documentation can be added to the site to talk about how you enable sqlite support after the fact.
Something wrong with it when running it different ways: I added SQLite to the project after created it!
go get -u -v -tags sqlite github.com/gobuffalo/pop/soda
go install -tags sqlite github.com/gobuffalo/pop/soda
#or may be rebuild buffalo itself with SQLite support
go get -u -v -tags sqlite github.com/gobuffalo/buffalo/buffalo
database.yml created with soda g config -t sqlite3
---
development:
dialect: "sqlite3"
database: /home/user/go/src/github.com/user/coke/sql_scripts/sqlite/coke.sqlite
test:
dialect: "sqlite3"
database: /home/user/go/src/github.com/user/coke/sql_scripts/sqlite/coke.sqlite
production:
dialect: "sqlite3"
database: /home/user/go/src/github.com/user/coke/sql_scripts/sqlite/coke.sqlite
buffalo-plugins.toml:
[[plugin]]
binary = "buffalo-plugins"
go_get = "github.com/gobuffalo/buffalo-plugins"
tags = ["sqlite"]
[[plugin]]
binary = "buffalo-pop"
go_get = "github.com/gobuffalo/buffalo-pop"
tags = ["sqlite"]
buffalo-app.toml:
name = "coke"
bin = "bin/coke"
vcs = "git"
with_pop = true
with_sqlite = true
with_dep = false
with_webpack = true
with_nodejs = true
with_yarn = true
with_docker = true
with_grifts = true
as_web = true
as_api = false
One line to log success and connection info (/models/models.go[func Init]):
func init() {
var err error
env := envy.Get("GO_ENV", "development")
DB, err = pop.Connect(env)
if err != nil {
log.Fatal(err)
}
pop.Debug = env == "development"
log.Println("Connection done: ", DB.String()) //<<<<<Report success connection
}
^^^^ ABOVE config are what I know about Buffalo and DB(pop/soda) if I missed sometihng correct me:
Afterward:
When running it with buffalo dev it seems works:
buffalo: 2019/12/27 20:58:01 === Building Completed (PID: 11243) (Time: 6.830132422s) ===
buffalo: 2019/12/27 20:58:01 === Running: tmp/coke-build (PID: 11356) ===
2019/12/27 20:58:03 Connection done: /home/user/go/src/github.com/user/coke/sql_scripts/sqlite/coke.sqlite?_busy_timeout=5000&_fk=true
INFO[2019-12-27T20:58:04+03:30] Starting application at http://127.0.0.1:3000
INFO[2019-12-27T20:58:04+03:30] Starting Simple Background Worker
BUT RUNNING IT WITH DEBUGGING MODE (GoLand| Run kind: Package|Package path: github.com/user/coke| Working directory:/home/user/go/src/github.com/user/coke)
will behave differently:
GOROOT=/USER_SPACE/Dev-Platform/go #gosetup
GOPATH=/home/user/go #gosetup
/USER_SPACE/Dev-Platform/go/bin/go build -o /tmp/___Debug___WebPack -gcflags "all=-N -l" github.com/user/coke #gosetup
/USER_SPACE/Applications/GoLand.2019.3/plugins/go/lib/dlv/linux/dlv --listen=localhost:35503 --headless=true --api-version=2 --check-go-version=false exec /tmp/___Debug___WebPack -- #gosetup
API server listening at: 127.0.0.1:35503
[POP] 2019/12/27 21:01:19 warn - unable to load connection development: unsupported dialect 'sqlite3'
[POP] 2019/12/27 21:01:19 warn - unable to load connection test: unsupported dialect 'sqlite3'
[POP] 2019/12/27 21:01:19 warn - unable to load connection production: unsupported dialect 'sqlite3'
2019/12/27 21:01:22 could not find connection named development
Debugger finished with exit code 0
ALSO
$ buffalo pop -d
v4.13.0
[POP] 2019/12/27 20:55:44 debug - Loading config file from /home/user/go/src/github.com/user/coke/database.yml
[POP] 2019/12/27 20:55:44 warn - unable to load connection test: unsupported dialect 'sqlite3'
[POP] 2019/12/27 20:55:44 warn - unable to load connection production: unsupported dialect 'sqlite3'
[POP] 2019/12/27 20:55:44 warn - unable to load connection development: unsupported dialect 'sqlite3'
As sugested here:
$ buffalo new newone --db-type=sqlite
Usage:
buffalo new [name] [flags]
Flags:
--api skip all front-end code and configure for an API server
--ci-provider string specify the type of ci file you would like buffalo to generate [none, travis, gitlab-ci] (default "none")
--config string config file (default is $HOME/.buffalo.yaml)
--db-type string specify the type of database you want to use [cockroach, mysql, postgres] (default "postgres")
--docker string specify the type of Docker file to generate [none, multi, standard] (default "multi")
-d, --dry-run dry run
-f, --force delete and remake if the app already exists
-h, --help help for new
--module string specify the root module (package) name. [defaults to 'automatic']
--skip-config skips using the config file
--skip-pop skips adding pop/soda to your app
--skip-webpack skips adding Webpack to your app
--skip-yarn use npm instead of yarn for frontend dependencies management
--vcs string specify the Version control system you would like to use [none, git, bzr] (default "git")
-v, --verbose verbosely print out the go get commands
ERRO[0000] Error: unknown dialect "sqlite" expecting one of cockroach, mysql, postgres
$ buffalo new newone --db-type=sqlite3
Usage:
buffalo new [name] [flags]
Flags:
--api skip all front-end code and configure for an API server
--ci-provider string specify the type of ci file you would like buffalo to generate [none, travis, gitlab-ci] (default "none")
--config string config file (default is $HOME/.buffalo.yaml)
--db-type string specify the type of database you want to use [cockroach, mysql, postgres] (default "postgres")
--docker string specify the type of Docker file to generate [none, multi, standard] (default "multi")
-d, --dry-run dry run
-f, --force delete and remake if the app already exists
-h, --help help for new
--module string specify the root module (package) name. [defaults to 'automatic']
--skip-config skips using the config file
--skip-pop skips adding pop/soda to your app
--skip-webpack skips adding Webpack to your app
--skip-yarn use npm instead of yarn for frontend dependencies management
--vcs string specify the Version control system you would like to use [none, git, bzr] (default "git")
-v, --verbose verbosely print out the go get commands
ERRO[0000] Error: unknown dialect "sqlite3" expecting one of cockroach, mysql, postgres
I also have the same problem when trying to run any tests using sqlite:
go test ./... or running test from VS Code
FAIL github.com/pplavetzki/coke/models 0.008s
[POP] 2021/06/07 13:04:21 warn - unable to load connection production: unsupported dialect 'sqlite3'
[POP] 2021/06/07 13:04:21 warn - unable to load connection development: unsupported dialect 'sqlite3'
[POP] 2021/06/07 13:04:21 warn - unable to load connection test: unsupported dialect 'sqlite3'
2021/06/07 13:04:21 could not find connection named development
Running in buffalo dev produces no errors and connects just fine to sqlite3.
database.yml:
---
development:
dialect: "sqlite3"
database: /home/pplavetzki/go/src/github.com/pplavetzki/coke/sqlite/coke_development.sqlite
test:
dialect: "sqlite3"
database: /home/pplavetzki/go/src/github.com/pplavetzki/coke/sqlite/coke_test.sqlite
production:
dialect: "sqlite3"
database: /home/pplavetzki/go/src/github.com/pplavetzki/coke/sqlite/coke_production.sqlite
buffalo-plugins.toml:
[[plugin]]
binary = "buffalo-pop"
go_get = "github.com/gobuffalo/buffalo-pop/v2"
tags = ["sqlite"]
buffalo-app.toml:
name = "coke"
bin = "bin/coke"
vcs = "git"
with_pop = true
with_sqlite = true
with_dep = false
with_webpack = true
with_nodejs = true
with_yarn = true
with_docker = true
with_grifts = true
as_web = true
as_api = false
Would love to know what the trick is to be able to run tests or debug without failing.
