DISCUSSION: Reduce amount of implicit magic

[dhermes]

See https://github.com/GoogleCloudPlatform/gcloud-python/pull/830/files#r29357515

[jgeewax]

IMO, the goal here should be to support the implicit magic if users want it (and hell, for a lot of people it's what they have come to expect from App Engine), however support the explicit side of things as well. Ultimately it should be easy to swap out credentials for the cases where we want them.

Disclaimer: This conversation may very well bring us in a full circle... and I apologize if that happens.

---

Some thoughts

1. <module>.<method>() calls (ie, datastore.get(...)) all would have a connection or client parameter.
2. If connection/client is None, we try to resolve it -- potentially using magic.
3. Setting a default client should have global effect (in other words, no per-module default client).
4. If you don't want a global default client, you should be able to easily create a client and then use it like you would use the module itself (ie pubsub_client.create_topic('topic_name')) (I'd expect this to be 'the recommended way' of doing things...)
5. A Client object would represent a combination of Credentials, a Project ID, and a transport, so a globally shared default client implies a globally shared default project ID as well as credentials.
6. Clients should have classmethod factories for loading credentials easily (ie, from_environment_variable, from_path, from_p12)

Using an explicitly built client object (not the module)

You'd use this whenever you use more than one set of credentials and one project ID (whether they're on the same service or not). The client object would just pass itself on as the argument and act as an intermediary:

class PubsubClient(Client):
    def create_topic(self, *args, **kwargs):
        pubsub.create_topic(connection=self.connection, *args, **kwargs)

Auto detect project ID and credentials (aka, don't think about it)

from gcloud import pubsub

pubsub_client = pubsub.Client()
pubsub_client.create_topic('topic_name')

Auto detect the project ID but specify where to get credentials

from gcloud import pubsub

pubsub_client = pubsub.Client(
    credentials=gcloud.Credentials.from_environment_variable('...'))
pubsub_client.create_topic('topic_name')

Auto detect the credentials but specify the project ID

from gcloud import pubsub

pubsub_client = pubsub.Client(project='project_id')
pubsub_client.create_topic('topic_name')

Specify both the project ID and the credentials

import gcloud
from gcloud import pubsub

pubsub_client = pubsub.Client(
    credentials=gcloud.Credentials.from_environment_variable('...'),
    project='project_id')
pubsub_client.create_topic('topic_name')

Using the module itself

You'd use this only when you have a single set of credentials and a single project.

Auto detect project ID and credentials (aka, don't think about it)

from gcloud import pubsub
pubsub.create_topic('topic_name')

Auto detect the project ID but specify where to get credentials

import gcloud
from gcloud import pubsub

gcloud.set_default_client(gcloud.Client(
    credentials=gcloud.Credentials.from_environment_variable('....')))
pubsub.create_topic('topic_name')

Auto detect the credentials but specify the project ID

import gcloud
from gcloud import pubsub

gcloud.set_default_client(gcloud.Client(project='project_id'))
pubsub.create_topic('topic_name')

Specify both the project ID and the credentials

import gcloud
from gcloud import pubsub

gcloud.set_default_client(gcloud.Client(
    credentials=gcloud.Credentials.from_environment_variable('....'),
    project='project_id'))
pubsub.create_topic('topic_name')

[jgeewax]

After our conversation yesterday, it seems like if our goal is to simplify the use of different "connection" objects we might be able to do this cleanly using the same type of pattern as we do with Batches by making the Connection object a context manager. This would make code look like:

connection = ...  # Get a Connection object somehow.
connection2 = ...  # Get a Connection object somehow.

with connection:
    pubsub.create_topic('topic_name')

with connection 2:
    pubsub.create_topic('topic_name')

We still need to address the problem of "If I want to swap out credentials, it should be easy and simple" -- in other words "Getting a connection object somehow." should be straight forward assuming the two things I want to tweak are the credentials and the project ID.

More discussion on this soon...

[dhermes]

This is kind of a redundant issue at this point, given #816, etc.