appengine:exec can't connect to Cloud SQL DB with private IP

[ktaras]

I have Rails application and Cloud SQL (PostgreSQL) database with Private IP. When I'm trying to run any task with appengine:exec I get PG::ConnectionBad error. I've created another database, but this time with public IP, and everything worked fine.

Here is the error:

rake aborted!
PG::ConnectionBad: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
/app/vendor/bundle/ruby/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `initialize'
/app/vendor/bundle/ruby/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `new'
/app/vendor/bundle/ruby/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `connect'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:692:in `connect'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:223:in `initialize'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-postgis-adapter-5.2.2/lib/active_record/connection_adapters/postgis_adapter.rb:62:in `initialize'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-postgis-adapter-5.2.2/lib/active_record/connection_adapters/postgis/create_connection.rb:40:in `new'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-postgis-adapter-5.2.2/lib/active_record/connection_adapters/postgis/create_connection.rb:40:in `postgis_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:811:in `new_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:855:in `checkout_new_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:834:in `try_to_checkout_new_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:795:in `acquire_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:523:in `checkout'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:382:in `connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:1010:in `retrieve_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/connection_handling.rb:90:in `connection'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/model_schema.rb:324:in `table_exists?'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/attribute_methods/primary_key.rb:99:in `get_primary_key'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/attribute_methods/primary_key.rb:87:in `reset_primary_key'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/attribute_methods/primary_key.rb:75:in `primary_key'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/relation/delegation.rb:76:in `primary_key'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/relation/batches.rb:274:in `batch_order'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/relation/batches.rb:217:in `in_batches'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/relation/batches.rb:69:in `find_each'
/app/vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.2/lib/active_record/querying.rb:11:in `find_each'
/app/config/sitemap.rb:24:in `block (2 levels) in run'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:61:in `instance_eval'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:61:in `eval'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/link_set.rb:240:in `group'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:45:in `group'
/app/config/sitemap.rb:21:in `block in run'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:61:in `instance_eval'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:61:in `eval'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/link_set.rb:40:in `create'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator.rb:41:in `method_missing'
/app/config/sitemap.rb:17:in `run'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:78:in `instance_eval'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/interpreter.rb:78:in `run'
/app/vendor/bundle/ruby/2.6.0/gems/sitemap_generator-6.0.2/lib/sitemap_generator/tasks.rb:51:in `block (2 levels) in <main>'
/app/vendor/bundle/ruby/2.6.0/gems/rake-12.3.2/exe/rake:27:in `<top (required)>'
/opt/rbenv/versions/2.6.1/bin/bundle:23:in `load'
/opt/rbenv/versions/2.6.1/bin/bundle:23:in `<main>'
Tasks: TOP => sitemap:refresh => sitemap:create
(See full trace by running task with --trace)
ERROR
ERROR: build step 0 "gcr.io/google-appengine/exec-wrapper:latest" failed: exit status 1

Is there any way to get around this issue? Can't find anything related in docs and source code.

[Houdini]

Same thing. Have you already found a solution?

[dazuma]

Unfortunately, It's not currently possible to connect to a private IP using appengine:exec. This is a limitation of the technique it uses. (It connects from Cloud Build VMs which are not within your app's VPC.)

There is a potential workaround (which I can make available in the upcoming 0.5 release) where we actually deploy a temporary version of your app to execute a command, but it does come with a significant performance penalty since App Engine flex deployments tend to be very slow.