ebean
ebean copied to clipboard
ebean-orm
application-test.yaml seems to be ignored when test makes second call to DatabaseConfig.loadFromProperties
Background
Often when testing in Spring, multiple application contexts can be created and destroyed. When using programatic configuration of ebeans this could create a situation where multiple calls are made to DatabaseConfig.loadFromProperties method.
Expected behavior
It doesn't matter how many times the tests call DatabaseConfig.loadFromProperties, the test configuration is found and used each time.
Actual behavior
Only the first call to DatabaseConfig.loadFromProperties succeeds. When trying to use a Database created from the DatabaseConfig that was created by a second call to DatabaseConfig.loadFromProperties we get exception:
io.ebean.datasource.DataSourceConfigurationException: DataSource user is not set? url is [null]
at app//io.ebean.datasource.pool.ConnectionPool.<init>(ConnectionPool.java:121)
at app//io.ebean.datasource.pool.ConnectionPoolFactory.createPool(ConnectionPoolFactory.java:14)
at app//io.ebean.datasource.DataSourceFactory.create(DataSourceFactory.java:25)
at app//io.ebeaninternal.server.core.InitDataSource.create(InitDataSource.java:113)
at app//io.ebeaninternal.server.core.InitDataSource.createFromConfig(InitDataSource.java:104)
at app//io.ebeaninternal.server.core.InitDataSource.initDataSource(InitDataSource.java:44)
at app//io.ebeaninternal.server.core.InitDataSource.initialise(InitDataSource.java:33)
at app//io.ebeaninternal.server.core.InitDataSource.init(InitDataSource.java:24)
at app//io.ebeaninternal.server.core.DefaultContainer.setDataSource(DefaultContainer.java:220)
at app//io.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:86)
at app//io.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:29)
at app//io.ebean.DatabaseFactory.createInternal(DatabaseFactory.java:136)
at app//io.ebean.DatabaseFactory.create(DatabaseFactory.java:85)
at app//org.example.PetTest.testDouble(PetTest.java:44)
It appears as though the test configuration isn't found.
Steps to reproduce
I've created a sample project with code at https://github.com/simontankersley/programatic-ebean-test
This code fails
// simulate using a custom object mapper
ObjectMapper mapper = new ObjectMapper();
// simulate creating a database configuration twice
// this often happens when using spring tests where several
// contexts can be created and destroyed - causing multiple
// calls to create the database configuration from properties
DatabaseConfig dbConfig = new DatabaseConfig();
dbConfig.loadFromProperties();
dbConfig.setObjectMapper(mapper);
Database db = DatabaseFactory.create(dbConfig);
dbConfig = new DatabaseConfig();
dbConfig.loadFromProperties();
dbConfig.setObjectMapper(mapper);
db = DatabaseFactory.create(dbConfig);
db.save(new Pet());
If you want to see a base line vs a failing test see below
Success when running a test with a single call to DatabaseConfig.loadFromProperties
git clone [email protected]:simontankersley/programatic-ebean-test.git
cd programatic-ebean-test
./gradlew test --tests '*testSingle'
gives
> Task :test
PetTest > testSingle() PASSED
Failure when running a test with a multiple calls to DatabaseConfig.loadFromProperties
./gradlew test --tests '*testDouble'
gives
> Task :test
PetTest > testDouble() FAILED
io.ebean.datasource.DataSourceConfigurationException: DataSource user is not set? url is [null]
at app//io.ebean.datasource.pool.ConnectionPool.<init>(ConnectionPool.java:121)
at app//io.ebean.datasource.pool.ConnectionPoolFactory.createPool(ConnectionPoolFactory.java:14)
at app//io.ebean.datasource.DataSourceFactory.create(DataSourceFactory.java:25)
at app//io.ebeaninternal.server.core.InitDataSource.create(InitDataSource.java:113)
at app//io.ebeaninternal.server.core.InitDataSource.createFromConfig(InitDataSource.java:104)
at app//io.ebeaninternal.server.core.InitDataSource.initDataSource(InitDataSource.java:44)
at app//io.ebeaninternal.server.core.InitDataSource.initialise(InitDataSource.java:33)
at app//io.ebeaninternal.server.core.InitDataSource.init(InitDataSource.java:24)
at app//io.ebeaninternal.server.core.DefaultContainer.setDataSource(DefaultContainer.java:220)
at app//io.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:86)
at app//io.ebeaninternal.server.core.DefaultContainer.createServer(DefaultContainer.java:29)
at app//io.ebean.DatabaseFactory.createInternal(DatabaseFactory.java:136)
at app//io.ebean.DatabaseFactory.create(DatabaseFactory.java:85)
at app//org.example.PetTest.testDouble(PetTest.java:44)
Thanks for the example project with failing test. I have cloned and reproduced the issue.
Some background: There are some considerations going on that we need to navigate.
- We are looking to start the appropriate docker containers once and set them up once (create the database, create the user/role etc)
- We are similarly looking to run the DDL once to drop and recreate the database schema
The current issue as I see it is that the "Run Once Marker" is covering those things that we do really only want to run once but it is also covering the function that does the "set up the DataSourceConfig given the test docker container" we want to run our tests against.
So I expect the fix for this issue to be along the lines of:
- moving that "Run Once Marker" to cover only the "setup the test docker containers"
- always executing the function that does the "set up the DataSourceConfig given the test docker container"
- adjust the DDL configuration such that we use DDL mode
nonefor subsequent executions (only run the DDL once)
Q: Is there a workaround in the meantime?
Probably look to use explicit url, username, password in application-test.yaml and programmatically set the appropriate ddl mode (which assumes we can determine when the first Ebean DatabaseFactory.create() is happening and setDdlRun(true) only for that creation).
Maybe workaround
A possible workaround is to explicitly specify the dataSource options in application-test.yaml like:
ebean:
test:
platform: postgres
dbName: test
ddlMode: dropCreate
datasource:
db:
username: test
password: test
url: jdbc:postgresql://localhost:6432/test
Noting that the tests now pass but we might not be getting what we want in terms of DDL being run. If we add logging of the test output (and adjust a little so we can see when the tests create the Database instances in the logs) we see:
$ ./gradlew clean test
> Task :compileJava
Note: Ebean APT generated 1 query beans, loaded 0 others - META-INF/ebean-generated-info.mf entity-packages: [org.example.model]
> Task :test
PetTest > atestSingle() STANDARD_OUT
23:23:50.577 [Test worker] TRACE io.avaje.config - load from [resource:application-test.yaml]
23:23:50.578 [Test worker] INFO io.avaje.config - Loaded properties from [resource:application-test.yaml]
23:23:50.579 [Test worker] DEBUG io.ebean.test - automatic testing config - with ebean.test.platform:postgres name:db environmentDb:null
23:23:50.583 [ForkJoinPool.commonPool-worker-2] INFO io.ebean.test - Using jdbc settings - username:test url:jdbc:postgresql://localhost:6432/test driver:org.postgresql.Driver
23:23:50.586 [ForkJoinPool.commonPool-worker-2] DEBUG io.ebean.test - Docker properties: {postgres.password=test, postgres.driver=org.postgresql.Driver, postgres.extensions=hstore,pgcrypto, postgres.username=test, postgres.version=14, postgres.url=jdbc:postgresql://localhost:6432/test, postgres.port=6432, postgres.dbName=test}
23:23:50.718 [ForkJoinPool.commonPool-worker-2] TRACE io.ebean.test.containers - sqlRun: select 1 from pg_database where datname = 'test'
23:23:50.726 [ForkJoinPool.commonPool-worker-2] DEBUG io.ebean.test.containers - Container ut_postgres ready with host:localhost port:6432
23:23:50.729 [Test worker] DEBUG io.ebean.test - for testing - using FixedEncryptKeyManager() keyVal:simple0123456789
23:23:50.729 [Test worker] INFO io.ebean.test - For testing purposes a current user provider has been configured. Use io.ebean.test.UserContext to set current user in tests.
23:23:50.729 [Test worker] INFO org.example.PetTest - testSingle ------------------------------
23:23:50.729 [Test worker] INFO io.ebean - ebean version: 13.15.0
23:23:50.777 [Test worker] INFO io.ebean.datasource - DataSource [db] autoCommit[false] transIsolation[READ_COMMITTED] min[2] max[200] in[40ms]
23:23:50.908 [Test worker] INFO io.ebean.DDL - Executing db-drop-all.sql - 1 statements, autoCommit:true
23:23:50.909 [Test worker] DEBUG io.ebean.DDL - executing 1 of 1 drop table if exists pet cascade
23:23:50.916 [Test worker] INFO io.ebean.DDL - Executing db-create-all.sql - 1 statements, autoCommit:false
23:23:50.917 [Test worker] DEBUG io.ebean.DDL - executing 1 of 1 create table pet ( id uuid not null, constraint p...
23:23:50.924 [Test worker] INFO io.ebean.core - Started database[db] platform[POSTGRES] in 192ms
23:23:50.927 [Test worker] DEBUG io.ebean.SQL - txn[1001] insert into pet (id) values (?); -- bind(ca527473-577c-418c-9028-8e667b58e4c3)
23:23:50.931 [Test worker] DEBUG io.ebean.TXN - txn[1001] Commit
PetTest > testTDouble() STANDARD_OUT
23:23:50.934 [Test worker] DEBUG io.ebean.test - automatic testing config - with ebean.test.platform:postgres name:db environmentDb:null
23:23:50.934 [Test worker] DEBUG io.ebean.test - for testing - using FixedEncryptKeyManager() keyVal:simple0123456789
23:23:50.934 [Test worker] INFO io.ebean.test - For testing purposes a current user provider has been configured. Use io.ebean.test.UserContext to set current user in tests.
23:23:50.934 [Test worker] INFO org.example.PetTest - testDouble 1 ------------------------------
23:23:50.967 [Test worker] INFO io.ebean.datasource - DataSource [db] autoCommit[false] transIsolation[READ_COMMITTED] min[2] max[200] in[32ms]
23:23:50.972 [Test worker] INFO io.ebean.core - Started database[db] platform[POSTGRES] in 37ms
23:23:50.972 [Test worker] DEBUG io.ebean.test - automatic testing config - with ebean.test.platform:postgres name:db environmentDb:null
23:23:50.972 [Test worker] DEBUG io.ebean.test - for testing - using FixedEncryptKeyManager() keyVal:simple0123456789
23:23:50.972 [Test worker] INFO io.ebean.test - For testing purposes a current user provider has been configured. Use io.ebean.test.UserContext to set current user in tests.
23:23:50.972 [Test worker] INFO org.example.PetTest - testDouble 2 ------------------------------
23:23:50.998 [Test worker] INFO io.ebean.datasource - DataSource [db] autoCommit[false] transIsolation[READ_COMMITTED] min[2] max[200] in[26ms]
23:23:51.002 [Test worker] INFO io.ebean.DDL - Executing db-drop-all.sql - 1 statements, autoCommit:true
23:23:51.002 [Test worker] DEBUG io.ebean.DDL - executing 1 of 1 drop table if exists pet cascade
23:23:51.005 [Test worker] INFO io.ebean.DDL - Executing db-create-all.sql - 1 statements, autoCommit:false
23:23:51.005 [Test worker] DEBUG io.ebean.DDL - executing 1 of 1 create table pet ( id uuid not null, constraint p...
23:23:51.008 [Test worker] INFO io.ebean.core - Started database[db] platform[POSTGRES] in 36ms
23:23:51.009 [Test worker] DEBUG io.ebean.SQL - txn[1001] insert into pet (id) values (?); -- bind(02036faf-fc6c-4f5c-96f2-fac25f06e1d3)
23:23:51.010 [Test worker] DEBUG io.ebean.TXN - txn[1001] Commit
23:23:51.016 [EbeanHook] INFO io.ebean.datasource - DataSource [db] shutdown min[2] max[200] free[2] busy[0] waiting[0] highWaterMark[1] waitCount[0] hitCount[4] psc[hit:0 miss:4 put:4 rem:0]
23:23:51.016 [EbeanHook] INFO io.ebean.datasource - DataSource [db] shutdown min[2] max[200] free[2] busy[0] waiting[0] highWaterMark[1] waitCount[0] hitCount[2] psc[hit:0 miss:1 put:1 rem:0]
23:23:51.017 [EbeanHook] INFO io.ebean.datasource - DataSource [db] shutdown min[2] max[200] free[2] busy[0] waiting[0] highWaterMark[1] waitCount[0] hitCount[4] psc[hit:0 miss:4 put:4 rem:0]
BUILD SUCCESSFUL in 1s
7 actionable tasks: 7 executed
... and we can see the ddl being executed multiple times. This might be ok but generally it is not good when we have a bigger schema (ddl takes time) or when we desire to run tests in parallel. Generally the desire is to drop and create the db schema once for all tests.