chore(provider): refactor sdk client creation to allow mock injection - #1663

Description

Intention

The intention of this refactoring is

• Being able to inject mocked SDK HTTP clients into resources. This makes unit-testing of resources easier.
  • Old approach (example): In the past we had to start a local mock server and use a custom endpoint to run the SDK requests against it. This doesn't allow for type safety during mocked tests, etc.
• New approach (example): We can inject a mocked client directly into the resource / datasource. We don't have to spin up mock servers and use custom endpoints. Furthermore we have type safety via our SDK mocks.

What changed inside the resource and datasource implementations

Previously our resources looked like this: In each resource in the Configure function a new SDK client was created. The client was created using a util function (boilerplate code which had to implemented for every service, also the tests were copied in boilerplate manner):

// Configure adds the provider configured client to the resource.
func (r *volumeResource) Configure(ctx context.Context, req resource.ConfigureRequest, resp *resource.ConfigureResponse) {
	var ok bool
	r.providerData, ok = conversion.ParseProviderData(ctx, req.ProviderData, &resp.Diagnostics)
	if !ok {
		return
	}

	apiClient := iaasUtils.ConfigureClient(ctx, &r.providerData, &resp.Diagnostics)
	if resp.Diagnostics.HasError() {
		return
	}
	r.client = apiClient
	tflog.Info(ctx, "iaas client configured")
}

Now instead we have a pre-populated struct core.ClientCollection which is part of the provider data and includes an instance of each SDK HTTP client.

// Configure adds the provider configured client to the resource.
func (r *volumeResource) Configure(ctx context.Context, req resource.ConfigureRequest, resp *resource.ConfigureResponse) {
	var ok bool
	providerData, clients, ok := core.ParseProviderData(ctx, req.ProviderData, &resp.Diagnostics)
	if !ok {
		return
	}

	r.providerData = providerData
	r.client = clients.IaaSv2Client

	tflog.Info(ctx, "iaas client configured")
}

How this works internally

First of all we have the core.ClientCollection struct which should be passed to every resource and datasource:

This client collection has to be initialized somehow with all the SDK clients. Therefore we have the initClientCollection function and the ClientFactory

The ClientFactory interface allows us to have a DefaultClientFactory implementation which is used in regular production use of the STACKIT Terraform provider.

The MockClientFactory is another implementation of the ClientFactory interface and allows us to inject mocked SDK clients into the provider for API-mocked testing of resource and datasource implementations.

The injection of the mock client factory happens here:

Splitting the provider data into two structs

In Terraform plugin framework you can have one struct which is passed as provider data to each resource and datasource.

In the past this struct looked like this:

It e.g. included a round tripper for authentication of all the SDK clients which were created using the util functions in each resource/datasource configure method.

Since to allow for easy mocking we need to pass pre-populated SDK clients to all resources/datasources in the future we need to add this client to provider data. The provider data struct would grow therefore massively with API clients for every STACKIT service and service version.

At the same we need to keep this provider data struct within our resource/datasource implementation structs to make the region handling possible (using a default_region configured on provider level and allowing our TF users to optionally override it in each resource/datasource configuration).

Reminder: We can only pass one provider data struct to each resource/datasource. Therefore I decided to split the provider data. The provider data struct passed should consist of two parts: The formerly known provider data (including e.g. the default_region config value or the list of enabled provider experiments) and the client collection.

The provider data struct is meant to be kept in each regional resource/datasource implementation in the same way we did until now. The client collection should only be used in the Configure method of each resource/datasource implementation.

Both structs are tied together using the core.providerDataInternal struct.

The providerDataInternal struct type isn't exposed by design from the core package. This way the inside the resource and datasource implementations the provider data stays the same like we've previously known it (it even shrinked since we don't need to keep all custom endpoint configs, etc. in it any more).

Ensuring code quality using automation

Two measures were taken to assert the code quality after this implementation:

• A tfclientcollection linter was implemented to make sure the core.ClientCollection struct is only consumed from core.ParseProviderData but stored in the resource/datasource implementation structs or passed as a parameter to other functions called by the resource/datasource Configure method implementations.
  • The core.ClientCollection struct can't be package-private in the core package because there's one exception to that: the generic IAM rolebinding resource/datasource implementations.
• A unit test was added to ensure each client available in the core.ClientCollection struct is actually initialized using the ClientFactory interface in the initClientCollection function: test

Furthermore for testing all the client initialization implementations of the DefaultClientFactory some generic test handler was implemented to get rid of the insane amount of boilerplate code we maintained beforehand.

Checklist

• Issue was linked above
• Code format was applied: make fmt
• Examples were added / adjusted (see examples/ directory)
• Docs are up-to-date: make generate-docs (will be checked by CI)
• Unit tests got implemented or updated
• Acceptance tests got implemented or updated (see e.g. here)
• Unit tests are passing: make test (will be checked by CI)
• No linter issues: make lint (will be checked by CI)