List all topics using pagination
Currently there's a limit of about 100 topics when calling Kane.Topic.all function, as the pubsub API paginates the results. There are several ways of removing this 100 topic limit, but the easiest one is, by default, traversing the next pages.
When listing topics, we get a map of the following kind:
%{
"topics" => list(%Kane.Topic{}),
"nextPageToken" => str
}
We can GET projects/#{project}/topics?pageToken=#{page_token} to get the next page. If we use this recursively we can get all of the tokens until the map no longer has the nextPageToken present and we conclude that's the last page.
Here's an example PR of how we can do this: https://github.com/peburrows/kane/pull/40
We can proceed on that idea in that same PR or we can move to a different one, I'm open to discuss ideas! That PR was tested with a pubsub with more than 100 topics and it worked without any issues.