Allow users to configure the behavior when a Key Vault reference cannot be resolved

[avanigupta]

Problem

The recommended way to load key-values from App Configuration is to specify only what your application needs to use in the Select clause(s). This can be achieved by name-spacing key-values with a prefix or placing key-values into separate labels and loading only the required labels.

This rule of thumb also applies to any Key Vault references that are stored in App Configuration. If your application loads a Key Vault reference, and you haven't configured the right credentials needed to fetch the secret value from Key Vault, an exception will be thrown. However, in certain situations, you may not be able to rely on filtering the Key Vault references and need more control over what happens after Key Vault references are fetched from App Configuration. Common scenarios when you might need a workaround for avoiding this exception:

• Your application doesn't need any secrets from Key Vault.
• Your application is configured to resolve some secrets from Key Vault, but you don't want your application to resolve any new Key Vault references that may be added to your App Config store in future.

Proposed Solution

Configuration provider should not implicitly hide the exception because that would leave the application configuration in an undefined state. In order to have a deterministic value for any unresolved Key Vault references, it's best that users explicitly control what they want to do when their application loads a Key Vault reference whose value should not be fetched from Key Vault.
For this, we can add a new API that allows users to set a callback which will be invoked to resolve Key Vault references that do not have a matching SecretClient registered.

public AzureAppConfigurationKeyVaultOptions SetSecretResolver(Func<Uri, ValueTask<string>> secretResolver)

This callback takes the Key Vault secret identifier (URI) as input and returns the value that will be added to your application configuration corresponding to the Key Vault reference key.

Usage Example

If your App Configuration has the following Key Vault references:

Key	Value
KeyVaultRef1	{"uri":"https://abc.vault.azure.net/secrets/SecretName"}
KeyVaultRef2	{"uri":"https://xyz.vault.azure.net/secrets/SecretName"}
KeyVaultRef3	{"uri":"https://unknown1.vault.azure.net/secrets/SecretName"}
KeyVaultRef4	{"uri":"https://unknown2.vault.azure.net/secrets/SecretName"}

The secret resolver callback can be configured as follows:

var builder = new ConfigurationBuilder();
builder.AddAzureAppConfiguration(options =>
{
    options.Connect(connectionString)
           .Select("KeyVaultRef*")
           .ConfigureKeyVault(kv =>
           {
               // Register a SecretClient for every Key Vault whose reference needs to be resolved
               kv.Register(new SecretClient(new Uri("https://abc.vault.azure.net"), new DefaultAzureCredential()));
               kv.Register(new SecretClient(new Uri("https://xyz.vault.azure.net"), new DefaultAzureCredential()));

               kv.SetSecretResolver((secretIdentifier) =>
               {
                   // Example: return the secret identifier to be used as value
                   return new ValueTask<string>(secretIdentifier.ToString());
               });
           });

Your application configuration will have the following key-values:

Key	Value
KeyVaultRef1	Secret Value from Key Vault 'abc'
KeyVaultRef2	Secret Value from Key Vault 'xyz'
KeyVaultRef3	https://unknown1.vault.azure.net/secrets/SecretName
KeyVaultRef4	https://unknown2.vault.azure.net/secrets/SecretName

Notes

• If you choose to set the secret resolver callback, you need to explicitly register all SecretClient(s) that your application needs for resolving Key Vault references.
• If the secret resolver callback is set, it will be used as a fallback mechanism to resolve Key Vault references when no matching SecretClient has been registered.
• You cannot set both secret resolver and default credentials as fallback - only one of the following can be invoked:
  • SetSecretResolver()
  • SetCredential()
• The secret resolver callback will not be invoked if you get a failure status code from one of the registered SecretClient(s).
• If you don't want to resolve any Key Vault references, you still need to call ConfigureKeyVault() and SetSecretResolver() in order to define the behavior you want for dealing with unresolvable Key Vault references.

cc: @abhilasharora @drago-draganov @jimmyca15 @zhenlan

[avanigupta]

Related customer issues: #139, #203

Hi @avanigupta, sorry for my late response, I've had little to no time to take a look at this the past weeks.
Thanks for proposing a change and tagging me.

I don't believe that this would work in our case, or at least, not easily. Our current code depends on SetCredential() to configure the credentials for all KeyVault references, without actually having to know their URIs, because they differ across multiple subscriptions and it would be difficult to define them beforehand.
We would rather have the ability to use both SetCredential() and SetSecretResolver(). Or, alternatively if that doesn't work, have a SetRegistryResolver() that basically lazily loads a SecretClient based on its URI the first time it's encountered. A.k.a. a resolver that accepts a Func<Uri, SecretClient> as parameter, or something similar. However, I don't know if this would be possible and/or feasible.

Actually, what I said above doesn't even work either because of

The secret resolver callback will not be invoked if you get a failure status code from one of the registered SecretClient(s).

Basically, our issue is that some KeyVault references may exist in AppConfiguration that some apps may not have access to, however at runtime, it's not known which KeyVaults they have or don't have access to. In other words, we would like to either have a callback when a failure status code is returned (so that we can intercept it and change the response), or completely ignore the KeyVault reference setting in case that happens.

[jimmyca15]

Hello @b10-dslappendel,

This solution provides complete control over the interaction between the App Configuration provider and Key Vault. Your application will have the capability to create SecretClients for separate key-vaults, caching them preferrably, and handle failures as you see fit.

You mentioned that you rely on SetCredential currently. That means, with this solution, that when the SecretClients are created, they should be created using the credential that you are passing to SetCredential currently.

This solution does not require knowing URIs ahead of time. The SecretClients can be created on demand, ensuring that.

[avanigupta]

Hi @b10-dslappendel, I agree with what @jimmyca15 has explained.

Basically, our issue is that some KeyVault references may exist in AppConfiguration that some apps may not have access to, however at runtime, it's not known which KeyVaults they have or don't have access to. In other words, we would like to either have a callback when a failure status code is returned (so that we can intercept it and change the response), or completely ignore the KeyVault reference setting in case that happens.

From the configuration provider's point of view, every key-value that is loaded by your application (through the Select clause) is possibly being used by your application. When a Key Vault secret cannot be resolved due to access issues, there are two possible situations:

1. Your app needs this Key Vault secret but does not have access to it - so eventually your app will crash when it tries to use this setting.
2. Your app does not care about this Key Vault secret and this error can be safely ignored.

I want to emphasize that there is no way for configuration provider to know which access errors are valid errors that cannot be ignored (i.e., the 1st situation), and which errors can be ignored (i.e., 2nd situation). That's why we want to make sure that if anyone is trying to implement something like this, it has to be done very explicitly in a way that gives the user full control over what finally goes in application configuration.

For your application, since you want to resolve secrets on the fly without having to track all the Key Vault URIs, you could implement your own secret resolver to fetch secrets from Key Vault with the appropriate exception handling. Here's a sample code snippet:

var builder = new ConfigurationBuilder();
builder.AddAzureAppConfiguration(options =>
{
    options.Connect(connectionString)
           .Select(KeyFilter.Any)
           .ConfigureKeyVault(kv =>
	        {
	            kv.SetSecretResolver(async (secretIdentifier) =>
	            {
	                // Example: Fetch secrets from KeyVault and return the secret value
	                string secretValue;

	                try
	                {
	                    string secretName = secretIdentifier?.Segments?.ElementAtOrDefault(2)?.TrimEnd('/');
	                    string secretVersion = secretIdentifier?.Segments?.ElementAtOrDefault(3)?.TrimEnd('/');
	                    SecretClient secretClient = new SecretClient(new Uri(secretIdentifier.GetLeftPart(UriPartial.Authority)), new DefaultAzureCredential());
	                    KeyVaultSecret secret = await secretClient.GetSecretAsync(secretName, secretVersion);
	                    secretValue = secret?.Value;
	                }
	                catch (UnauthorizedAccessException)
	                {
	                    // Handle the Key Vault reference value when this app doesn't have access to Key Vault
	                    secretValue = "DummyValue";
	                }

	                return secretValue;
	            });
	        });
});

@avanigupta, ohh, I completely missed the fact that it's possible to handle secret retrieval manually this way. Many thanks for the snippet, I'll take a look and see if I can get it working!

Update - I've managed to get it working with the latest commit. I'll now be waiting eagerly for v4.1. Thanks again 😀

[avanigupta]

@b10-dslappendel ,
This feature has been released in package version 4.1.0:

• Microsoft.Extensions.Configuration.AzureAppConfiguration
• Microsoft.Azure.AppConfiguration.AspNetCore