early table name serialization prevents sharding with switchman

[anthuswilliams]

We have a Rails app that does sharding using https://github.com/instructure/switchman, an ActiveRecord extension which manages database sharding by using several PostgreSQL schemas and switching between them appropriately when executing ActiveRecord queries.

closure_tree does not work at all with switchman because it serializes the hierarchies table name at load time, when it is included in an ActiveRecord model. This happens in the default shard (the "public" schema). Then, when we switch shards in order to execute a closure_tree query in a different schema, it incorrectly tries to pull records from "public.#{ table }_hierarchies", rather than "#{ schema }.#{ table }_hierarchies".

In order to help see what I'm talking about, here is a monkey-patch that works around this problem in our app:

https://gist.github.com/anthuswilliams/bce19169ae1fa531d798765031bc320d

I thought I'd introduce this issue in case anyone is trying something similar, and on the off chance we can work up a solution within closure_tree. In the meantime, I guess no Rails codebase is complete without a monkey-patch or two.

[mceachen]

Interesting—is the sharding library changing table names, or just schema names?

It'd be a biggish patch, but I guess every table name reference could be prefixed with a schema-name-providing lambda to fully qualify everything. And the default could be an empty string.

Would that work?

[anthuswilliams]

It just changes schema names (and in some cases, the database server on which the query will be run), but I don't think it necessarily needs to accept a proc. I'm pretty sure all that would need to happen is to delay construction of the order by clause (https://github.com/mceachen/closure_tree/blob/6a67e3a8e402d27a36553d1611654155c6ac1f48/lib/closure_tree/model.rb#L14) until the query actually executes.

For context, switchman adds a use_qualified_names option to database.yml, which, when set, overrides quoted_table_name to prepend the schema of the active shard. In pseudocode, it looks something like this:

shard = Switchman::Shard.find(<shard id>)
shard.activate do
  # execute a query involving closure_tree here
  # if we were to generate the sql here, rather than at load time, 
  #  quoted_table_name will refer to the correct schema
end