Skip to content

Commit

Permalink
Create core package and refactor data label + datamap (#439)
Browse files Browse the repository at this point in the history
* Refactor data label

* Proposed structure

* Finish draft

* Fix comment

* Functional datalabel package

* Stable version

* Improve docs

* Address reviewer comments

* Refactor dependencies and organize namespaces

* Stable version

* Update docs

* Draft package names

* Stable version: all packages refactored

* Fix docs examples

* Avoid recreating datamap due to API inconsistency

* Sync up docs with latest changes in code

* Add reference to  docs in main README
  • Loading branch information
wcmjunior authored Nov 3, 2023
1 parent df710e6 commit 4427cd3
Show file tree
Hide file tree
Showing 158 changed files with 3,902 additions and 3,197 deletions.
17 changes: 7 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ Full and comprehensive documentation for this provider with detailed description

Please refer to our [Change Log](CHANGELOG.md) to learn about our version history, its features, bug fixes and Control Plane compatibility.

## Building, Documenting and Testing this Project
## Building, Maintaining, Documenting and Testing this Project

### Build Instructions

In order to build this repository, follow the steps below:

1. Clone [terraform-provider-cyral](https://github.com/cyralinc/terraform-provider-cyral) repo from GitHub;

2. Go to the root directory of the cloned repo using Linux shell and execute `make`. The build process will create binaries in directory `out` for both `darwin` and `linux` 64 bits. These binaries will be copied automatically to the local Terraform registry to be used by Terraform 13 and later.
2. Go to the root directory of the cloned repo using Linux shell and execute `make`. The build process will create binaries in directory `out` for both `darwin` and `linux` 64 bits. These binaries will be copied automatically to the local Terraform registry to be used by Terraform 0.13 and later.

Alternatively, you can use the dockerfile to build the image using `make docker-compose/build`

Expand All @@ -36,6 +36,11 @@ terraform {
}
```

### Adding new resources or data sources

Use the abstractions provided in the package `core` to add new resources or data sources.
Read the documentation in [`cyral.core`](./cyral/core/README.md) for more information.

### Updating the Documentation

This project uses [`terraform-plugin-docs`](https://github.com/hashicorp/terraform-plugin-docs).
Expand Down Expand Up @@ -104,12 +109,4 @@ pre-commit install

### Running Project Built Locally

#### Terraform 0.12

Copy the desired binary file created in directory `out` (see [Build Instructions](#build-instructions)) to the root folder containing those `.tf` files that will be used to handle Cyral Terraform provider resources.

Run `terraform init` and proceed with `terraform apply` normally to execute your Terraform scripts.

#### Terraform 0.13+

Build the project using steps in [Build Instructions](#build-instructions), then proceed normally with `terraform init` and `terraform apply` commands.
52 changes: 48 additions & 4 deletions client/client.go → cyral/client/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import (
"io/ioutil"
"log"
"net/http"
"os"
"strconv"
"strings"

"golang.org/x/oauth2"
Expand All @@ -16,6 +18,13 @@ import (

const redactedString = "**********"

const (
EnvVarClientID = "CYRAL_TF_CLIENT_ID"
EnvVarClientSecret = "CYRAL_TF_CLIENT_SECRET"
EnvVarCPURL = "CYRAL_TF_CONTROL_PLANE"
EnvVarTLSSkipVerify = "CYRAL_TF_TLS_SKIP_VERIFY"
)

// Client stores data for all existing resources. Also, this is
// the struct that is passed along resources CRUD operations.
type Client struct {
Expand All @@ -24,9 +33,9 @@ type Client struct {
client *http.Client
}

// NewClient configures and returns a fully initialized Client.
func NewClient(clientID, clientSecret, controlPlane string, tlsSkipVerify bool) (*Client, error) {
log.Printf("[DEBUG] Init NewClient")
// New configures and returns a fully initialized Client.
func New(clientID, clientSecret, controlPlane string, tlsSkipVerify bool) (*Client, error) {
log.Printf("[DEBUG] Init client.New")

if clientID == "" || clientSecret == "" || controlPlane == "" {
return nil, fmt.Errorf("clientID, clientSecret and controlPlane must have non-empty values")
Expand All @@ -49,7 +58,7 @@ func NewClient(clientID, clientSecret, controlPlane string, tlsSkipVerify bool)
tokenSource := tokenConfig.TokenSource(context.Background())

log.Printf("[DEBUG] TokenSource: %v", tokenSource)
log.Printf("[DEBUG] End NewClient")
log.Printf("[DEBUG] End client.New")

return &Client{
ControlPlane: controlPlane,
Expand Down Expand Up @@ -141,3 +150,38 @@ func redactContent(content string) string {
}
return redactedString
}

func FromEnv() (*Client, error) {
clientID, clientSecret, controlPlane, tlsSkipVerify, err :=
getProviderConfigFromEnv()
if err != nil {
return nil, fmt.Errorf("unable to create Cyral client: %w", err)
}
c, err := New(clientID, clientSecret, controlPlane,
tlsSkipVerify)
if err != nil {
return nil, fmt.Errorf("unable to create Cyral client: %w", err)
}
return c, nil
}

func getProviderConfigFromEnv() (
clientID string,
clientSecret string,
controlPlane string,
tlsSkipVerify bool,
err error,
) {
clientID = os.Getenv(EnvVarClientID)
clientSecret = os.Getenv(EnvVarClientSecret)
controlPlane = os.Getenv(EnvVarCPURL)
tlsSkipVerifyStr := os.Getenv(EnvVarTLSSkipVerify)
if tlsSkipVerifyStr != "" {
tlsSkipVerify, err = strconv.ParseBool(tlsSkipVerifyStr)
if err != nil {
return "", "", "", false, fmt.Errorf("invalid value for "+
"env var %q: %w", EnvVarTLSSkipVerify, err)
}
}
return
}
10 changes: 5 additions & 5 deletions client/client_test.go → cyral/client/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ func TestNewClient_WhenTLSSkipVerifyIsEnabled_ThenInsecureSkipVerifyIsTrue(t *te
controlPlane := "someControlPlane"
tlsSkipVerify := true

client, err := NewClient(clientID, clientSecret, controlPlane, tlsSkipVerify)
client, err := New(clientID, clientSecret, controlPlane, tlsSkipVerify)

require.NoError(t, err)

Expand All @@ -37,7 +37,7 @@ func TestNewClient_WhenTLSSkipVerifyIsDisabled_ThenInsecureSkipVerifyIsFalse(t *
controlPlane := "someControlPlane"
tlsSkipVerify := false

client, err := NewClient(clientID, clientSecret, controlPlane, tlsSkipVerify)
client, err := New(clientID, clientSecret, controlPlane, tlsSkipVerify)

require.NoError(t, err)

Expand All @@ -59,7 +59,7 @@ func TestNewClient_WhenClientIDIsEmpty_ThenThrowError(t *testing.T) {
controlPlane := "someControlPlane"
tlsSkipVerify := false

client, err := NewClient(clientID, clientSecret, controlPlane, tlsSkipVerify)
client, err := New(clientID, clientSecret, controlPlane, tlsSkipVerify)

expectedErrorMessage := "clientID, clientSecret and controlPlane must have non-empty values"

Expand All @@ -73,7 +73,7 @@ func TestNewClient_WhenClientSecretIsEmpty_ThenThrowError(t *testing.T) {
controlPlane := "someControlPlane"
tlsSkipVerify := false

client, err := NewClient(clientID, clientSecret, controlPlane, tlsSkipVerify)
client, err := New(clientID, clientSecret, controlPlane, tlsSkipVerify)

expectedErrorMessage := "clientID, clientSecret and controlPlane must have non-empty values"

Expand All @@ -87,7 +87,7 @@ func TestNewClient_WhenControlPlaneIsEmpty_ThenThrowError(t *testing.T) {
controlPlane := ""
tlsSkipVerify := false

client, err := NewClient(clientID, clientSecret, controlPlane, tlsSkipVerify)
client, err := New(clientID, clientSecret, controlPlane, tlsSkipVerify)

expectedErrorMessage := "clientID, clientSecret and controlPlane must have non-empty values"

Expand Down
File renamed without changes.
File renamed without changes.
201 changes: 201 additions & 0 deletions cyral/core/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# Cyral Provider Core

The `core` package was created in order to put together all the code that is responsible
for managing the Provider itself and to provide reusable functions and abstractions
for resources and data sources.

## How to use to create new resources and data sources

There are some main types that must be used to create a new resources and data sources:
`SchemaDescriptor`, `PackageSchema`, `ResourceData`, `ResponseData` and
`ResourceOperationConfig`. In a nutshell, these abstractions provide the means to
teach the provider how to interact with the API, how to describe the feature as a
Terraform resource/data source and finally teach the provider how to perform the
translation from API to Terraform schema and vice-versa.

Use the files below as examples to create your own implementation. It is advised that
you follow the same naming convention for all the files to simplify future code changes.

### model.go

```go
// model.go
package newfeature

type NewFeature struct {
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
}

func (r *NewFeature) WriteToSchema(d *schema.ResourceData) error {
if err := d.Set("description", r.Description); err != nil {
return fmt.Errorf("error setting 'description' field: %w", err)
}
d.SetId(r.Name)
return nil
}

func (r *NewFeature) ReadFromSchema(d *schema.ResourceData) error {
r.Name = d.Get("name").(string)
r.Description = d.Get("description").(string)
return nil
}
```

### datasource.go

```go
// datasource.go
package newfeature

func dataSourceSchema() *schema.Resource {
return &schema.Resource{
Description: "Some description.",
ReadContext: core.ReadResource(core.ResourceOperationConfig{
Name: "NewFeatureRead",
HttpMethod: http.MethodGet,
CreateURL: func(d *schema.ResourceData, c *client.Client) string {
return fmt.Sprintf("https://%s/v1/NewFeature/%s", c.ControlPlane, d.Get("name").(string))
},
NewResponseData: func(d *schema.ResourceData) core.ResponseData {
return &NewFeature{}
},
}),
Schema: map[string]*schema.Schema{
"name": {
Description: "Retrieve the unique label with this name, if it exists.",
Type: schema.TypeString,
Optional: true,
},
"description": {
Description: "Description of the data source.",
Type: schema.TypeString,
Optional: true,
},
},
}
}
```

### resource.go

```go
// resource.go
package newfeature

func resourceSchema() *schema.Resource {
return &schema.Resource{
Description: "Some description.",
CreateContext: core.CreateResource(
core.ResourceOperationConfig{
Name: "NewFeatureResourceRead",
HttpMethod: http.MethodPost,
CreateURL: func(d *schema.ResourceData, c *client.Client) string {
return fmt.Sprintf("https://%s/v1/NewFeature", c.ControlPlane)
},
NewResponseData: func(d *schema.ResourceData) core.ResponseData {
return &NewFeature{}
},
}, ReadNewFeatureConfig,
),
ReadContext: core.ReadResource(ReadNewFeatureConfig),
UpdateContext: core.UpdateResource(
core.ResourceOperationConfig{
Name: "NewFeatureUpdate",
HttpMethod: http.MethodPut,
CreateURL: func(d *schema.ResourceData, c *client.Client) string {
return fmt.Sprintf("https://%s/v1/NewFeature/%s", c.ControlPlane, d.Id())
},
NewResourceData: func() core.ResourceData { return &NewFeature{} },
}, ReadNewFeatureConfig,
),
DeleteContext: core.DeleteResource(
core.ResourceOperationConfig{
Name: "NewFeatureDelete",
HttpMethod: http.MethodDelete,
CreateURL: func(d *schema.ResourceData, c *client.Client) string {
return fmt.Sprintf("https://%s/v1/NewFeature/%s", c.ControlPlane, d.Id())
},
},
),
Schema: map[string]*schema.Schema{
"name": {
Description: "...",
Type: schema.TypeString,
Optional: true,
},
"description": {
Description: "...",
Type: schema.TypeString,
Optional: true,
},
},
}
}

var ReadNewFeatureConfig = core.ResourceOperationConfig{
Name: "NewFeatureRead",
HttpMethod: http.MethodGet,
CreateURL: func(d *schema.ResourceData, c *client.Client) string {
return fmt.Sprintf("https://%s/v1/NewFeature/%s", c.ControlPlane, d.Id())
},
NewResponseData: func(_ *schema.ResourceData) core.ResponseData { return &NewFeature{} },
RequestErrorHandler: &core.ReadIgnoreHttpNotFound{ResName: "NewFeature"},
}
```

### schema_loader.go

```go
// schema_loader.go
package newfeature

type packageSchema struct {
}

func (p *packageSchema) Name() string {
return "newfeature"
}

func (p *packageSchema) Schemas() []*core.SchemaDescriptor {
return []*core.SchemaDescriptor{
{
Name: "cyral_newfeature",
Type: core.DataSourceSchemaType,
Schema: dataSourceSchema,
},
{
Name: "cyral_newfeature",
Type: core.ResourceSchemaType,
Schema: resourceSchema,
},
}
}

func PackageSchema() core.PackageSchema {
return &packageSchema{}
}
```

### provider/schema_loader.go

Edit the existing `cyral/provider/schema_loader.go` file and add your new package schema
to function `packagesSchemas` as follows:

```go
package provider

import (
"github.com/cyralinc/terraform-provider-cyral/cyral/core"
...
"github.com/cyralinc/terraform-provider-cyral/cyral/internal/newfeature"
)

func packagesSchemas() []core.PackageSchema {
v := []core.PackageSchema{
...,
newfeature.PackageSchema(),
}
return v
}
```
10 changes: 10 additions & 0 deletions cyral/core/constants.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package core

type OperationType string

const (
OperationTypeCreate = OperationType("create")
OperationTypeRead = OperationType("read")
OperationTypeUpdate = OperationType("update")
OperationTypeDelete = OperationType("delete")
)
Loading

0 comments on commit 4427cd3

Please sign in to comment.