DevOps BootCamp

Material for 8 hours Practical Immersion with Golang This is a material in Golang to be presented "face-to-face" in a "hand in hand" Workshop that will be done in 8 hours.

Golang Workshop DevOps

All content aims at the basic level of the student many practical examples were made with details richness to make life easier than it is initiating.

If you know little and almost nothing programming will not be problem every manual was made to level starting to advanced. All the difficulties I had when I started trying to contemplate this material. We will try to improve the material all the time so we can have a reference when it comes to Go.

I hope you all enjoy it and can serve as a base for learning and help several possible Gophers.

The content and references used are from the Golang Official Site and the material being developed which is a compilation of all Golang language and can be checked here jeffotoni/Compilation.

Some presentations I made can be viewed here Presentations.

There are thousands of references today regarding Golang, let's start at the beginning and we could not stop talking about Golang Tour. Well that site here Play Golang or Play Go Space we can play Golang online.

We have a very interesting link that we have been able to search for packages written in Golang check out this link: Go Doc

We have this link that presents us as a manual all libs developed in Golang Dev Docs

Here we find an awesome go, there are several lists as it is, and sometimes it's cool to check out some libs to help us with some projects. awesome-go

Soon below some channels that I participate and can find me online.

Telegram:

gobr

Slack:

gophers.slack.com
- Brazil
- Brasil
- General
- Go-kit
- Gotimefm

Lab 01 Install and Commands Golang

Overview
Introduction Installation
- Installation
Installation Docker
Introduction Golang
- Golang Language
Go Commands
- Go Commands Introduction
- Go Run
- Go Build
- Go Install
- Go Get
- Go Mod
- Go Mod Init
- Go Mod Vendor
- GO111MODULE
- Go Test

Lab 03 Parse with Golang, Yaml, Toml and Json

Json
- Introduction
Parse Json
Links Json to Golang
Exercise Three

Lab 04 Building apis with net/http

Introduction http
Exercise Five
net/http Client
- Introduction
Exercise Six
net/http Server Pages
- Introduction
Exercise Seven

Lab 05 Using Golang to Create Command Line Programs

Golang Cli
- Introduction
Exercise Five

Lab 06 Goroutine The Power

Overview

Go is a powerful language when it comes to competition and high performance, with a clean and efficient architecture. It grows year after year and every day the communities grow even more.

Some paradigms have been broken to make it a high performance language, where competition is one of its strengths. Go makes it easy to create programs that take full advantage of multicore and networked machines, while the new type system allows you to build flexible and modular programs.

It is a fast and statically compiled language that looks like a dynamically interpreted language. This feature Golang becomes a unique language as the subject is web.

Go is a compiled, competing, strong and statically typed programming language. It is a "General Use" language that can be used to solve various problems and in different areas. Problems involving competition, web applications, high performance applications, development of APIs, communications sockets etc ... Is where language is increasingly becoming prominent in the market and in communities.

Introduction Installation

In golang the installation is all very simple and practical, for Linux, Mac and Windows.

Just copy the files to the correct directory for each operating system and export the paths to the environment and prompt, golang is installed.

Let's take a look at how we do this.

Installation

We will download the file, unpack it and install it in /usr/local/go, if we have golang already installed in the machine we will have to remove the existing one to leave our installation as unique. Let's create our directory in our workspace and test to see if everything went well

Linux

$ sudo rm -rf /usr/local/go
$ wget https://dl.google.com/go/go1.11.5.linux-amd64.tar.gz
$ sudo tar -C /usr/local -xzf go$VERSION.$OS-$ARCH.tar.gz

$GOPATH

$GOPATH is the golang in your $HOME, this is necessary for your projects to use pkg and build properly. This was mandatory for all versions before version 1.11. The cool thing is that from now on we will not have to create projects in $GOPATH, we can create in any other directory that is not in $GOPATH.

Here is the link to the versioning proposal Proposal: Versioned Go Modules or Go 1.11 Modules

We'll detail how to work with go mod, it was one of the best experiences I had for versioning projects using Golang.

Let's set up our environment to run Go. Add /usr/local/go/bin to the PATH environment variable. You can do this by adding this line to your /etc/profile (for a system-wide installation) or $HOME/.profile.

$ export PATH=$PATH:/usr/local/go/bin

Note: changes made to a profile file may not apply until the next time you log into your computer. To apply the changes immediately, just run the shell commands directly or execute them from the profile using a command such as source $HOME/.profile.

$ echo "export GOPATH=$HOME/go" >> $HOME/.profile
$ echo "export PATH=$PATH:/usr/local/go/bin" >> $HOME/.profile
$ echo "export PATH=$PATH:$GOPATH/bin" >> $HOME/.profile

Test our Installation

Let's run go version to see if everything is correct.

$ go version
go version go1.11.5 linux/amd64

Check that Go is installed correctly by setting up a workspace and building a simple program, as follows.

Create your workspace directory, $HOME/go. (If you'd like to use a different directory, you will need to set the $GOPATH environment variable.)

Next, make the directory src/hello inside your workspace, and in that directory create a file named hello.go that looks like:

Workspace

Workspace is our place of work, where we will organize our directories with our projects. As shown above, until Go version 1.11 we were forced to do everything under the Workspace. $GOPATH Down Projects.

Example Hello

$ export GOPATH=$HOME/go
$ mkdir $HOME/go
$ mkdir $HOME/go/src
$ mkdir $HOME/go/src/hello
$ vim $HOME/go/src/hello/hello.go

$GOPATH/
  |-src
    |-hello
      |-hello.go

Example Project

$ export GOPATH=$HOME/go
$ mkdir $HOME/go/src/project1
$ mkdir $HOME/go/src/project1/my-pkg
$ mkdir $HOME/go/src/project1/my-cmd
$ mkdir $HOME/go/src/project1/my-vendor
$ mkdir $HOME/go/src/project1/my-logs
$ mkdir $HOME/go/src/project1/my-models
$ mkdir $HOME/go/src/project1/my-repo
$ mkdir $HOME/go/src/project1/my-handler

$GOPATH/
  |-src
    |-github.com/user/project1/
        |-cmd (of project1)
          |-main.go
        |-vendor
        |-logs
        |-models
        |-repo
        |-handler
    |-github.com/user/project2/
      ....
      ....

The $GOPATH environment variable tells the Go tool where your workspace is located.

$ go get github.com/user/project1

The go get command fetches source repositories from the internet and places them in your workspace. Package paths matter to the Go tool. Using "github.com/..." means the tool knows how to fetch your repository.

In the scenario above everything would have to stay in our $GOPATH so that our projects worked correctly.

Outside $GOPATH

Now we can do our projects without being in $GOPATH, we can, for example, do it in any directory.

Project Outside GOPATH

$ export GOPATH=$HOME/go
$ mkdir $HOME/2019/project1
$ mkdir $HOME/2019/project1/my-pkg
$ mkdir $HOME/2019/project1/my-cmd
$ mkdir $HOME/2019/project1/my-logs
$ mkdir $HOME/2019/project1/my-models
$ mkdir $HOME/2019/project1/my-repo
$ mkdir $HOME/2019/project1/my-handler

$HOME/
  |-2019
    |-github.com/user/project1/
      |-cmd
        |-main.go
      |-vendor
      |-logs
      |-models
      |-repo
      |-handler

We can put our project in any directory now.

$HOME/
  |-any-directory
    |-github.com/user/project1/
      |-cmd
        |-main.go
      |-vendor
      |-logs
      |-models
      |-repo
      |-handler

For the above scenario, we will have to use go mod in our project so that all external packages can work correctly, in this way we will be able to manage them correctly and version. More information can be found here: Wiki Go Modules

Practical example of how you will proceed:

$ go mod init github.com/user/project1

Note: When we use go mod in $GOPATH we will have to enable using GO111MODULE=on, so that it can work within the $GOPATH structure. So our program can compile successfully.

$ GO111MODULE=on go run cmd/main.go
$ GO111MODULE=on go build -o project1 cmd/main.go

Installation Docker

If we do not want to install directly on our operating system golang we can install it in a docker container.

We can upload a docker container with the installed language and compile and run our programs from this container.

Let's check how we can do it below.

More information and details you can visit this link: hub.docker

Install Docker to Golang

$ docker pull golang

Compile Your app Inside The Docker Container

There may be occasions where it is not appropriate to run your app inside a container. To compile, but not run your app inside the Docker instance, you can write something like:

$ docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.11.5 go build -v

This will add your current directory as a volume to the container, set the working directory to the volume, and run the command go build which will tell go to compile the project in the working directory and output the executable to myapp. Alternatively, if you have a Makefile, you can run the make command inside your container.

$ docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.11.5 make

Cross-compile Your app Inside The Docker Container

If you need to compile your application for a platform other than linux/amd64 (such as windows/386):

$ docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp -e GOOS=windows \
-e GOARCH=386 golang:1.11.5 go build -v

Example main.go

Let's do our test program, let's call it main.go

package main

import "fmt"

func main(){
	fmt.Println("My first program being compiled by a docker container!")
}

Now let's run a program to see if it works correctly

$ docker run --rm -v "$PWD":/usr/src/main -w /usr/src/main golang:1.11.5 go run main.go

Output:

My first program being compiled by a docker container!

Check the version:

$ docker run --rm -v "$PWD":/usr/src/main -w /usr/src/main golang:1.11.5 go versio

Output:

go version go1.11.5 linux/amd64

Introduction Golang

Go is a general-purpose language designed with systems programming in mind. It is strongly typed and garbage-collected and has explicit support for concurrent programming. Programs are constructed from packages, whose properties allow efficient management of dependencies.

The grammar is compact and regular, allowing for easy analysis by automatic tools such as integrated development environments.

Golang Language

Keywords

The following keywords are reserved and may not be used as identifiers.

break        default      func         interface    select
case         defer        go           map          struct
chan         else         goto         package      switch
const        fallthrough  if           range        type
continue     for          import       return       var

Operators and Punctuation

The following character sequences represent operators (including assignment operators) and punctuation:

+    &     +=    &=     &&    ==    !=    (    )
-    |     -=    |=     ||    <     <=    [    ]
*    ^     *=    ^=     <-    >     >=    {    }
/    <<    /=    <<=    ++    =     :=    ,    ;
%    >>    %=    >>=    --    !     ...   .    :
     &^          &^=

Println Print

Let's learn how to send data to screen which is actually stdout standard output we will see more ahead with details on stdout and stdin.

Let's know print, println and fmt.Println

Current implementations provide several built-in functions useful during bootstrapping. These functions are documented for completeness but are not guaranteed to stay in the language. They do not return a result.

Implementation restriction: print and println need not accept arbitrary argument types, but printing of boolean, numeric, and string types must be supported.

println is an built-in function (into the runtime) which may eventually be removed, while the fmt package is in the standard library, which will persist.

Function   Behavior

print      prints all arguments; formatting of arguments is implementation-specific
println    like print but prints spaces between arguments and a newline at the end

using print:

// test print
package main

func main() {
   print("debugging my system with print")
}

Output:

debugging my system with print

using println:

// test println
package main

func main() {
   println("debugging my system with println")
}

Output:

debugging my system with println

using fmt.Println:

package main

import "fmt"

func main() {
   fmt.Println("debugging my system with fmt.Println")
}

Output:

debugging my system with fmt.Println

The goal of starting and running the print, println or fmt.Println command is to help us with the tests we will be performing from now on at every step of our Go learning.

Bufio NewWriter

bufio.Writer

Doing many small writes can hurt performance. Each write is ultimately a syscall and if doing frequently can put burden on the CPU. Devices like disks work better dealing with block-aligned data. To avoid the overhead of many small write operations Golang is shipped with bufio.Writer. Data, instead of going straight to destination (implementing io.Writer interface) are first accumulated inside the buffer and send out when buffer is full:

Let’s visualise how buffering works with nine writes (one character each) when buffer has space for 4 characters:

producer         buffer           destination (io.Writer)
 
   a    ----->   a
   b    ----->   ab
   c    ----->   abc
   d    ----->   abcd
   e    ----->   e      ------>   abcd
   f    ----->   ef               abcd
   g    ----->   efg              abcd
   h    ----->   efgh             abcd
   i    ----->   i      ------>   abcdefgh

Check out the example below

package main

import (
	"bufio"
	"os"
)

// creating the write object pointer
// so that we can receive value in every
// scope of our program
var writer *bufio.Writer

func main() {
	// All screen output will be redirected
	// to bufio.NewWriter
	writer = bufio.NewWriter(os.Stdout)
	s := "How many stars does Orion have?\n"
	var b byte = 'H'

	writer.WriteString(s)
	writer.WriteByte(b)
	writer.WriteString("\n")

	// when all the functions finishes it closes
	// the buffer and sends to the.Stdout
	defer writer.Flush()
}

Output:

How many stars does Orion have?
H

Func Main

package main

import "fmt"

func main() {
  fmt.Printf("hello, Gophers\n")
}

Then build it with the go tool:

$ cd $HOME/go/src/hello
$ go build

Or we can compile like this:

$ cd $HOME/go/src/hello
$ go build -o hello hello.go

The command above will build an executable named hello in the directory alongside your source code. Execute it to see the greeting:

$ ./hello
hello, Gophers

Check also the command run it with the go:

$ go run hello.go
hello, Gophers

If you see the "hello, Gophers" message then your Go installation is working.

You can run go install to install the binary into your workspace's bin directory or go clean -i to remove it.

Example: go install

$ pwd
$ $HOME/go/src/hello
$ cd $HOME/go/src/hello
$ go install
$ ls -lhs $HOME/go/bin
-rwxrwxr-x 1 user user 2,9M nov  8 03:11 hello

Example: go clean -i

$ go clean -i 
$ ls -lhs $HOME/go/bin

Go Commands

Go Commands Introduction

In golang we have an arsenal to help us when it comes to compiling, testing, documenting, managing Profiling etc.

bug         start a bug report
build       compile packages and dependencies
clean       remove object files and cached files
doc         show documentation for package or symbol
env         print Go environment information
fix         update packages to use new APIs
fmt         gofmt (reformat) package sources
generate    generate Go files by processing source
get         download and install packages and dependencies
install     compile and install packages and dependencies
list        list packages or modules
mod         module maintenance
run         compile and run Go program
test        test packages
tool        run specified go tool
version     print Go version
vet         report likely mistakes in packages

Use "go help " for more information about a command.

Go Run

Usage:

go run [build flags] [-exec xprog] package [arguments...]

Run compiles and runs the named main Go package. Typically the package is specified as a list of .go source files, but it may also be an import path, file system path, or pattern matching a single known package, as in 'go run .' or 'go run my/cmd'.

By default, 'go run' runs the compiled binary directly: 'a.out arguments...'. If the -exec flag is given, 'go run' invokes the binary using xprog:

If the -exec flag is not given, GOOS or GOARCH is different from the system default, and a program named go_$GOOS_$GOARCH_exec can be found on the current search path, 'go run' invokes the binary using that program, for example 'go_nacl_386_exec a.out arguments...'. This allows execution of cross-compiled programs when a simulator or other execution method is available.

The exit status of Run is not the exit status of the compiled binary.

For more about build flags, see 'go help build'. For more about specifying packages, see 'go help packages'.

See below an example:

// test println
package main

func main() {
   println("Debugging my system with println")
}

Go run:

go run println.go

Output:

Debugging my system with println

Go Build

Build compiles the packages named by the import paths, along with their dependencies, but it does not install the results.

When compiling packages, build ignores files that end in '_test.go'.

The -o flag, only allowed when compiling a single package, forces build to write the resulting executable or object to the named output file, instead of the default behavior described in the last two paragraphs.

The -i flag installs the packages that are dependencies of the target.

$ go build [-o output] [-i] [build flags] [packages]

See an example:

package main

import "fmt"

func main() {
  fmt.Println("Workshop Golang 2019.")
}

Output:

Workshop Golang 2019.

Normal compilation

go build -o hello hello.go

Output:

$ ls -lh 
-rwxrwxr-x 1 root root **1,9M** jan 18 12:42 hello
-rw-rw-r-- 1 root root   75 jan 17 12:04 hello.go

Leaving the file smaller after compiling

go build -ldflags="-s -w" hello.go

Output:

$ ls -lh 
-rwxrwxr-x 1 root root **1,3M** jan 18 12:42 hello
-rw-rw-r-- 1 root root   75 jan 17 12:04 hello.go

Go Install

Install packages and dependencies

Usage:

$ go install [-i] [build flags] [packages]

Install compiles and installs the packages named by the import paths.

The -i flag installs the dependencies of the named packages as well.

For more about the build flags, see 'go help build'. For more about specifying packages, see 'go help packages'.

Go Get

The 'go get' command changes behavior depending on whether the go command is running in module-aware mode or legacy GOPATH mode. This help text, accessible as 'go help module-get' even in legacy GOPATH mode, describes 'go get' as it operates in module-aware mode.

Usage:

$ go get [-d] [-m] [-u] [-v] [-insecure] [build flags] [packages]

Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'.

Look at the flags accepted below:

The -d flag instructs get to stop after downloading the packages; that is, it instructs get not to install the packages.

The -f flag, valid only when -u is set, forces get -u not to verify that each package has been checked out from the source control repository implied by its import path. This can be useful if the source is a local fork of the original.

The -fix flag instructs get to run the fix tool on the downloaded packages before resolving dependencies or building the code.

The -insecure flag permits fetching from repositories and resolving custom domains using insecure schemes such as HTTP. Use with caution.

The -t flag instructs get to also download the packages required to build the tests for the specified packages.

The -u flag instructs get to use the network to update the named packages and their dependencies. By default, get uses the network to check out missing packages but does not use it to look for updates to existing packages.

The -v flag enables verbose progress and debug output.

Examples:

$ go get -v github.com/guptarohit/asciigraph
$ go get -u github.com/mxk/go-sqlite
$ go get -v github.com/google/uuid
$ go get -v github.com/sirupsen/logru

Go Mod

A module is a collection of related Go packages. Modules are the unit of source code interchange and versioning. The go command has direct support for working with modules, including recording and resolving dependencies on other modules. Modules replace the old GOPATH-based approach to specifying which source files are used in a given build.

Usage:

$ go mod <command> [arguments]

A module is defined by a tree of Go source files with a go.mod file in the tree's root directory. The directory containing the go.mod file is called the module root. Typically the module root will also correspond to a source code repository root (but in general it need not). The module is the set of all Go packages in the module root and its subdirectories, but excluding subtrees with their own go.mod files.

The "module path" is the import path prefix corresponding to the module root. The go.mod file defines the module path and lists the specific versions of other modules that should be used when resolving imports during a build, by giving their module paths and versions.

For example, this go.mod declares that the directory containing it is the root of the module with path example.com/m, and it also declares that the module depends on specific versions of golang.org/x/text and gopkg.in/yaml.v2:

$ go mod init github.com/user/gomyproject

require (
  golang.org/x/text v0.3.0
  gopkg.in/yaml.v2 v2.1.0
)

The go.mod file can also specify replacements and excluded versions that only apply when building the module directly; they are ignored when the module is incorporated into a larger build. For more about the go.mod file, see 'go help go.mod'.

To start a new module, simply create a go.mod file in the root of the module's directory tree, containing only a module statement. The 'go mod init' command can be used to do this:

$ go mod init github.com/user/gomyproject

In a project already using an existing dependency management tool like godep, glide, or dep, 'go mod init' will also add require statements matching the existing configuration.

Once the go.mod file exists, no additional steps are required: go commands like 'go build', 'go test', or even 'go list' will automatically add new dependencies as needed to satisfy imports.

The commands are:

download    download modules to local cache
edit        edit go.mod from tools or scripts
graph       print module requirement graph
init        initialize new module in current directory
tidy        add missing and remove unused modules
vendor      make vendored copy of dependencies
verify      verify dependencies have expected content
why         explain why packages or modules are needed

Use "go help mod " for more information about a command.

Go Mod Init

Initialize new module in current directory

Usage:

$ go mod init [module]

Init initializes and writes a new go.mod to the current directory, in effect creating a new module rooted at the current directory. The file go.mod must not already exist. If possible, init will guess the module path from import comments (see 'go help importpath') or from version control configuration. To override this guess, supply the module path as an argument.

$ go mod init github.com/user/gomyproject2

require (
  github.com/dgrijalva/jwt-go v3.2.0+incompatible
  github.com/didip/tollbooth v4.0.0+incompatible
  github.com/go-sql-driver/mysql v1.4.1
  github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
  golang.org/x/crypto v0.0.0-20190103213133-ff983b9c42bc
  golang.org/x/time v0.0.0-20181108054448-85acf8d2951c // indirect
)

Go Mod Vendor

The go mod vendor command will download all dependencies to the "vendor" directory. When using go mod init the packages are not in your directory.

$ cd gomyproject2
$ go mod vendor

Output:

$ ls -lh vendor
total 8,0K
drwxrwxr-x 3 root root 4,0K jan 27 01:47 github.com
-rw-rw-r-- 1 root root  137 jan 27 01:47 modules.txt

GO111MODULE

Go 1.11 includes preliminary support for Go modules, including a new module-aware 'go get' command. We intend to keep revising this support, while preserving compatibility, until it can be declared official (no longer preliminary), and then at a later point we may remove support for work in GOPATH and the old 'go get' command.

The quickest way to take advantage of the new Go 1.11 module support is to check out your repository into a directory outside GOPATH/src, create a go.mod file (described in the next section) there, and run go commands from within that file tree.

For more fine-grained control, the module support in Go 1.11 respects a temporary environment variable, GO111MODULE, which can be set to one of three string values: off, on, or auto (the default). If GO111MODULE=off, then the go command never uses the new module support. Instead it looks in vendor directories and GOPATH to find dependencies; we now refer to this as "GOPATH mode." If GO111MODULE=on, then the go command requires the use of modules, never consulting GOPATH. We refer to this as the command being module-aware or running in "module-aware mode". If GO111MODULE=auto or is unset, then the go command enables or disables module support based on the current directory. Module support is enabled only when the current directory is outside GOPATH/src and itself contains a go.mod file or is below a directory containing a go.mod file.

In module-aware mode, GOPATH no longer defines the meaning of imports during a build, but it still stores downloaded dependencies (in GOPATH/pkg/mod) and installed commands (in GOPATH/bin, unless GOBIN is set).

Check below how we use the command:

$ GO111MODULE=on go run myprogram.go
$ GO111MODULE=on go build myprogram.go

When our project is not in our $GOPATH it is not necessary to use GO111MODULE, but when our project is in $GOPATH and we want to use "go mod" we need to inform this to the compiler using GO111MODULE...

Go Test

Test packages

Usage:

$ go test [build/test flags] [packages] [build/test flags & test binary flags]

Go test automates testing the packages named by the import paths. It prints a summary of the test results in the format:

=== RUN   TestWhatever
--- PASS: TestWhatever (0.00s)
PASS
ok    command-line-arguments  0.001s

The test package runs side-by-side with the go test command. The package test should have the suffix "_test.go". We can split the tests into several files following this convention. For example: "myprog1_test.go" and "myprog2_test.go".

We should put our test functions in these test files.

Each test function is an exported public function whose name begins with "Test", accepts a pointer to a testing.T object, and returns nothing. Like this:

Example one / myprog1_test:

package main

import "testing"

func TestWhatever(t *testing.T) {
    // Your test code goes here
}

$ go test -v

Output:

=== RUN   TestWhatever
--- PASS: TestWhatever (0.00s)
PASS
ok    command-line-arguments  0.001s

The T object provides several methods that we can use to indicate failures or log errors.

Example two / myprog2_test:

package main

import "testing"

func TestSum(t *testing.T) {
  x := 1 + 1
  if x != 11 { // forcing the error
    t.Error("Error! 1 + 1 is not equal to 2, I got", x)
  }
}

$ go test -v

Output:

=== RUN   TestSum
-- FAIL: TestSum (0.00s)
    myprog1_test.go:12: Error! 1 + 1 is not equal to 2, I got 2
FAIL
FAIL  command-line-arguments  0.001s

In this example we will make an examination as it would be in our projects.

In this program I will pass parameter at compile time or in our execution to facilitate and also serve as an example the use of "ldflags" that we can use in both **go run -ldflags ** and go build -ldflags

From a check in our code below / main.go:

import "strconv"

import (
  "github.com/jeffotoni/goworkshopdevops/examples/tests/pkg/math"
)

var (
  x, y   string
  xi, yi int
)

func init() {
  xi, _ = strconv.Atoi(x)
  yi, _ = strconv.Atoi(y)
}

func main() {
  println(math.Sum(xi, yi))
}

Now we have a Sum function in a pkg that we create in pkg/math/sum.go

package math

func Sum(x, y int) int {
  return x + y
}

We created our test file in pkg/math/sum_test.go

package math

import "testing"

func TestSum(t *testing.T) {
  type args struct {
    x int
    y int
  }
  tests := []struct {
    name string
    args args
    want int
  }{
    // TODO: Add test cases.
    {"test_1: ", args{2, 2}, 4},
    {"test_2: ", args{-2, 6}, 4},
    {"test_3: ", args{-4, 8}, 4},
    {"test_4: ", args{5, 7}, 12},
    {"test_5: ", args{8, 8}, 15}, // forcing the error
  }
  for _, tt := range tests {
    t.Run(tt.name, func(t *testing.T) {
      if got := Sum(tt.args.x, tt.args.y); got != tt.want {
        t.Errorf("Sum() = %v, want %v", got, tt.want)
      }
    })
  }
}

$ cd pkg/math/
$ go test -v

Output:

=== RUN   TestSum
=== RUN   TestSum/test_1:_
=== RUN   TestSum/test_2:_
=== RUN   TestSum/test_3:_
=== RUN   TestSum/test_4:_
=== RUN   TestSum/test_5:_
--- FAIL: TestSum (0.00s)
    --- PASS: TestSum/test_1:_ (0.00s)
    --- PASS: TestSum/test_2:_ (0.00s)
    --- PASS: TestSum/test_3:_ (0.00s)
    --- PASS: TestSum/test_4:_ (0.00s)
    --- FAIL: TestSum/test_5:_ (0.00s)
        sum_test.go:29: Sum() = 16, want 15
FAIL
exit status 1
FAIL  github.com/jeffotoni/goworkshopdevops/examples/tests/pkg/pkg/math  0.001s

It converts to json the output of the tests

$ go test -v -json

check your output on your terminal screen to view json output.

Now that we've saved our pkg / math / sum.go let's do a main.go by making the call in this packet. But first let's run go mod to manage our packages and versions correctly.

Check the command below:

$ go mod init github.com/jeffotoni/goworkshopdevops/examples/tests/pkg

Output:

go: finding github.com/jeffotoni/goworkshopdevops/examples/tests/pkg/math latest
go: finding github.com/jeffotoni/goworkshopdevops/examples/tests latest
go: finding github.com/jeffotoni/goworkshopdevops/examples latest
go: finding github.com/jeffotoni/goworkshopdevops latest
go: downloading github.com/jeffotoni/goworkshopdevops v0.0.0-20190127023912-a2fa53299867
0

Now we can do build or run in our main.go. Let's run go run using the "-ldflags" flag to pass parameter to our code at compile time.

$ go run -ldflags "-X main.x=2 -X main.y=3" main.go

Output:

$ go run -ldflags "-X main.x=7 -X main.y=3" main.go

Output:

Lab 02 Types with Golang

Types

A type determines a set of values together with operations and methods specific to those values. A type may be denoted by a type name, if it has one, or specified using a type literal, which composes a type from existing types.

The language predeclares certain type names. Others are introduced with type declarations. Composite types—array, struct, pointer, function, interface, slice, map, and channel types—may be constructed using type literals.

Each type T has an underlying type: If T is one of the predeclared boolean, numeric, or string types, or a type literal, the corresponding underlying type is T itself. Otherwise, T's underlying type is the underlying type of the type to which T refers in its type declaration.

Example:

type (
    A1 = string
    A2 = A1
)

type (
    B1 string
    B2 B1
    B3 []B1
    B4 B3
)

The underlying type of string, A1, A2, B1, and B2 is string. The underlying type of []B1, B3, and B4 is []B1.

Numeric Types

A numeric type represents sets of integer or floating-point values. The predeclared architecture-independent numeric types are:

uint8       the set of all unsigned  8-bit integers (0 to 255)
uint16      the set of all unsigned 16-bit integers (0 to 65535)
uint32      the set of all unsigned 32-bit integers (0 to 4294967295)
uint64      the set of all unsigned 64-bit integers (0 to 18446744073709551615)

int8        the set of all signed  8-bit integers (-128 to 127)
int16       the set of all signed 16-bit integers (-32768 to 32767)
int32       the set of all signed 32-bit integers (-2147483648 to 2147483647)
int64       the set of all signed 64-bit integers (-9223372036854775808 to 9223372036854775807)

float32     the set of all IEEE-754 32-bit floating-point numbers
float64     the set of all IEEE-754 64-bit floating-point numbers

complex64   the set of all complex numbers with float32 real and imaginary parts
complex128  the set of all complex numbers with float64 real and imaginary parts

byte        alias for uint8
rune        alias for int32

The value of an n-bit integer is n bits wide and represented using two's complement arithmetic.

There is also a set of predeclared numeric types with implementation-specific sizes:

uint     either 32 or 64 bits
int      same size as uint
uintptr  an unsigned integer large enough to store the uninterpreted bits of a pointer value

To avoid portability issues all numeric types are defined types and thus distinct except byte, which is an alias for uint8, and rune, which is an alias for int32. Conversions are required when different numeric types are mixed in an expression or assignment. For instance, int32 and int are not the same type even though they may have the same size on a particular architecture.

String Types

A string type represents the set of string values. A string value is a (possibly empty) sequence of bytes. Strings are immutable: once created, it is impossible to change the contents of a string. The predeclared string type is string; it is a defined type.

The length of a string s (its size in bytes) can be discovered using the built-in function len. The length is a compile-time constant if the string is a constant. A string's bytes can be accessed by integer indices 0 through len(s)-1. It is illegal to take the address of such an element; if s[i] is the i'th byte of a string, &s[i] is invalid.

package main

import "fmt"

type S string

var (
  String = "@jeffotoni"
)

func main() {
  var text string
  var str S

  mypicture := "@Photograph-jeffotoni"
  str = "@workshop-devOpsBh"
  text = "@jeffotoni-golang"

  fmt.Println(str)
  fmt.Println(String)
  fmt.Println(text)
  fmt.Println(mypicture)

  // example string
  s := "日本語"
  fmt.Printf("Glyph:             %q\n", s)
  fmt.Printf("UTF-8:             [% x]\n", []byte(s))
  fmt.Printf("Unicode codepoint: %U\n", []rune(s))
}

Output:

@workshop-devOpsBh
@jeffotoni
@jeffotoni-golang
@Photograph-jeffotoni
Glyph:             "日本語"
UTF-8:             [e6 97 a5 e6 9c ac e8 aa 9e]
Unicode codepoint: [U+65E5 U+672C U+8A9E]

Pointer Types

Struct fields can be accessed through a struct pointer.

To access the field X of a struct when we have the struct pointer p we could write (*p).X. However, that notation is cumbersome, so the language permits us instead to write just p.X, without the explicit dereference.

A pointer type denotes the set of all pointers to variables of a given type, called the base type of the pointer. The value of an uninitialized pointer is nil.

PointerType = "*" BaseType .
BaseType    = Type .

*Point
*[4]int

Example:

package main

import "fmt"

type Vertex struct {
  X int
  Y int
}

func main() {
  v := Vertex{1, 2}
  p := &v
  p.X = 1e9
  fmt.Println(v)
  fmt.Println(p.Y)
}

Output:

{1000000000 2}
2

For every type that is declared, either by you or the language itself, you get for free a complement pointer type you can use for sharing. There already exists a built-in type named int so there is a complement pointer type called *int.

All pointer types have the same two characteristics. First, they start with the character *. Second, they all have the same memory size and representation, which is a 4 or 8 bytes that represent an address. On 32bit architectures (like the playground), pointers require 4 bytes of memory and on 64bit architectures (like your machine), they require 8 bytes of memory.

Example:

package main

func main() {

  var a int
  inc := &a
  *inc = 2
  *inc++
  println("inc:\tValue Of[", inc, "]\tAddr Of[", &inc, "]\tValue Points To[", *inc, "]")
}

Output:

inc:  Value Of[ 0xc000036778 ]  Addr Of[ 0xc000036780 ]  Value Points To[ 3 ]

For an operand x of type T, the address operation &x generates a pointer of type *T to x. The operand must be addressable, that is, either a variable, pointer indirection, or slice indexing operation; or a field selector of an addressable struct operand; or an array indexing operation of an addressable array. As an exception to the addressability requirement, x may also be a (possibly parenthesized) composite literal. If the evaluation of x would cause a run-time panic, then the evaluation of &x does too.

For an operand x of pointer type *T, the pointer indirection *x denotes the variable of type T pointed to by x. If x is nil, an attempt to evaluate *x will cause a run-time panic.

&x
&a[f(2)]
&Point{2, 3}
*p
*pf(x)

var x *int = nil
*x   // causes a run-time panic
&*x  // causes a run-time panic

See the example below:

package main

import "fmt"

func main() {
  var a int = 100  /* actual variable declaration */
  var pa *int      /* pointer variable declaration */
  var pointer *int /* pointer variable declaration */

  pa = &a /* store address of a in pointer variable*/

  fmt.Printf("Address of a variable: %x\n", &a)

  /* address stored in pointer variable */
  fmt.Printf("Address stored in ip variable: %x\n", pa)

  /* access the value using the pointer */
  fmt.Printf("Value of *ip variable: %d\n", *pa)

  /* succeeds if p is not nil */
  if pa != nil {
    fmt.Println("success is not nil")
  }

  /* succeeds if p is null */
  if (pointer) == nil {
    fmt.Println("success pointer is nil")
  }
}

Output:

Address of a variable: c0000160c8
Address stored in ip variable: c0000160c8
Value of *ip variable: 100
success is not nil
success pointer is nil

Array Types

An array is a numbered sequence of elements of a single type, called the element type. The number of elements is called the length and is never negative.

ArrayType   = "[" ArrayLength "]" ElementType .
ArrayLength = Expression .
ElementType = Type .

The length is part of the array's type; it must evaluate to a non-negative constant representable by a value of type int. The length of array a can be discovered using the built-in function len. The elements can be addressed by integer indices 0 through len(a)-1. Array types are always one-dimensional but may be composed to form multi-dimensional types.

[32]byte
[2*N] struct { x, y int32 }
[1000]*float64
[3][5]int
[2][2][2]float64  // same as [2]([2]([2]float64))

An array's length is part of its type, so arrays cannot be resized. This seems limiting, but don't worry; Go provides a convenient way of working with arrays.

Example:

package main

import "fmt"

func main() {

  // var a []string // wrong

  // An array of 10 integers
  var a1 [5]int
  a1[0] = 5
  a1[1] = 4
  a1[2] = 3
  a1[3] = 2
  a1[4] = 1
  fmt.Println(a1)
}

Output:

[5 4 3 2 1]

Slice Types

A slice is a descriptor for a contiguous segment of an underlying array and provides access to a numbered sequence of elements from that array. A slice type denotes the set of all slices of arrays of its element type. The value of an uninitialized slice is nil.

An array has a fixed size. A slice, on the other hand, is a dynamically-sized, flexible view into the elements of an array. In practice, slices are much more common than arrays.

The type []T is a slice with elements of type T.

A slice is formed by specifying two indices, a low and high bound, separated by a colon:

a[low : high]

This selects a half-open range which includes the first element, but excludes the last one.

The following expression creates a slice which includes elements 1 through 3 of a:

a[1:4]

Example:

package main

import "fmt"

func main() {
  primes := [7]int{2, 3, 5, 7, 11, 13, 14}
  
  var p []int = primes[2:5]
  fmt.Println(p)
}

Output:

[5 7 11]

A new, initialized slice value for a given element type T is made using the built-in function make, which takes a slice type and parameters specifying the length and optionally the capacity. A slice created with make always allocates a new, hidden array to which the returned slice value refers. That is, executing.

make([]T, length, capacity)

Produces the same slice as allocating an array and slicing it, so these two expressions are equivalent:

make([]int, 50, 100)
new([100]int)[0:50]

Like arrays, slices are always one-dimensional but may be composed to construct higher-dimensional objects. With arrays of arrays, the inner arrays are, by construction, always the same length; however with slices of slices (or arrays of slices), the inner lengths may vary dynamically. Moreover, the inner slices must be initialized individually.

Slices can be created with the built-in make function; this is how you create dynamically-sized arrays.

The make function allocates a zeroed array and returns a slice that refers to that array:

a := make([]int, 4)  // len(a)=4

Example:

package main

import "fmt"

func main() {
  a := make([]int,4)
  a[0]=12
  fmt.Println("a", a)

  b := make([]int, 0, 5)
  fmt.Println("b", b)
  
  c := b[:2]
  fmt.Println("c", c)
}

Output:

a [12 0 0 0]
b []
c [0 0]

A slice of type T is declared using []T. For example, Here is how you can declare a slice of type int -

// Slice of type `int`
var slice []int

// Slice of type `string`
var slice []string

// Slice of type `string` with parameter variadic
var lang = [...]string{"Erlang", "Elixir", "Haskell", "Clojure", "Scala"}

You can create a slice using a slice literal like this -

// Creating a slice using a slice literal
var s = []int{3, 5, 7, 9, 11, 13, 17}

The expression on the right-hand side of the above statement is a slice literal. The slice literal is declared just like an array literal, except that you do not specify any size in the square brackets [].

When you create a slice using a slice literal, it first creates an array and then returns a slice reference to it.

Let’s see:

package main

import "fmt"

func main() {
	// Creating a slice using a slice literal
	var s = []string{"@jeffotoni", "@awsbrasil", "@devopsbh", "@go_br"}

	// Short hand declaration
	t := []int{2, 4, 8, 16, 32, 64}

	fmt.Println("s = ", s, len(s))
	fmt.Println("t = ", t, len(t))
}

Output:

s =  [@jeffotoni @awsbrasil @devopsbh @go_br] 4
t =  [2 4 8 16 32 64] 6

Since a slice is a segment of an array, we can create a slice from an array.

To create a slice from an array a, we specify two indices low (lower bound) and high (upper bound) separated by a colon

// Obtaining a slice from an array `a`
a[low:high]

The above expression selects a slice from the array a. The resulting slice includes all the elements starting from index low to high, but excluding the element at index high.

package main

import "fmt"

func main() {
	// Creating a slice using a slice literal
	var groups = [5]string{"@awsbrasil", "@devopsbh", "@go_br", "@devopsbr", "@docker"}

	// Creating a slice from the array
	var s []string = groups[2:5]

	s2 := s[1:3]
	s3 := s[:3]
	s4 := s[2:]
	s5 := s[:]

	fmt.Println("Array groups = ", groups, "len:", len(groups), "cap:", cap(groups))
	fmt.Println("Slice s = ", s, "len:", len(s), "cap:", cap(s))
	fmt.Println("Slice s = ", s2, "len:", len(s2), "cap:", cap(s2))
	fmt.Println("Slice s = ", s3, "len:", len(s3), "cap:", cap(s3))
	fmt.Println("Slice s = ", s4, "len:", len(s4), "cap:", cap(s4))
	fmt.Println("Slice s = ", s5, "len:", len(s5), "cap:", cap(s5))
}

Output:

Array groups =  [@awsbrasil @devopsbh @go_br @devopsbr @docker] len: 5 cap: 5
Slice s =  [@go_br @devopsbr @docker] len: 3 cap: 3
Slice s =  [@devopsbr @docker] len: 2 cap: 2
Slice s =  [@go_br @devopsbr @docker] len: 3 cap: 3
Slice s =  [@docker] len: 1 cap: 1
Slice s =  [@go_br @devopsbr @docker] len: 3 cap: 3

The copy() function copies elements from one slice to another. Its signature looks like this

func copy(dst, src []T) int

It takes two slices - a destination slice, and a source slice. It then copies elements from the source to the destination and returns the number of elements that are copied.

Note that the elements are copied only if the destination slice has sufficient capacity.

package main

import "fmt"

func main() {

	src := []string{"Erlang", "Elixir", "Haskell", "Clojure", "Scala"}
	dest := make([]string, 2)

	numElementsCopied := copy(dest, src)

	fmt.Println("src = ", src)
	fmt.Println("dest = ", dest)
	fmt.Println("Number of elements copied from src to dest = ", numElementsCopied)
}

Output:

src =  [Erlang Elixir Haskell Clojure Scala]
dest =  [Erlang Elixir]
Number of elements copied from src to dest =  2

The append() function appends new elements at the end of a given slice. Following is the signature of append function.

func append(s []T, x ...T) []T

It takes a slice and a variable number of arguments x …T. It then returns a new slice containing all the elements from the given slice as well as the new elements.

If the given slice doesn’t have sufficient capacity to accommodate new elements then a new underlying array is allocated with bigger capacity. All the elements from the underlying array of the existing slice are copied to this new array, and then the new elements are appended.

However, if the slice has enough capacity to accommodate new elements, then the append() function re-uses its underlying array and appends new elements to the same array.

Let’s see an example:

package main

import "fmt"

func main() {
	slice1 := []string{"Clojure", "Scala", "Elixir"}
	slice2 := append(slice1, "Assembly", "Rust", "Go")
	fmt.Printf("slice1 = %v, len = %d, cap = %d\n", slice1, len(slice1), cap(slice1))
	fmt.Printf("slice2 = %v, len = %d, cap = %d\n", slice2, len(slice2), cap(slice2))
	slice1[0] = "C++"
	fmt.Println("\nslice1 = ", slice1)
	fmt.Println("slice2 = ", slice2)
	
	// slice nil
	var s []string

	// Appending to a nil slice
	s = append(s, "Java", "C", "Lisp", "Haskell")
	fmt.Printf("\ns = %v, len = %d, cap = %d\n", s, len(s), cap(s))
}

Output:

slice1 = [Clojure Scala Elixir], len = 3, cap = 3
slice2 = [Clojure Scala Elixir Assembly Rust Go], len = 6, cap = 6

slice1 =  [C++ Scala Elixir]
slice2 =  [Clojure Scala Elixir Assembly Rust Go]

s = [Java C Lisp Haskell], len = 4, cap = 4

Struct Types

A struct is a sequence of named elements, called fields, each of which has a name and a type. Field names may be specified explicitly (IdentifierList) or implicitly (EmbeddedField). Within a struct, non-blank field names must be unique.

StructType    = "struct" "{" { FieldDecl ";" } "}" .
FieldDecl     = (IdentifierList Type | EmbeddedField) [ Tag ] .
EmbeddedField = [ "*" ] TypeName .
Tag           = string_lit .

// An empty struct.
struct{}

// A struct with 6 fields.
struct {
  x, y int
  u float32
  _ float32  // padding
  A *[]int
  F func()
}

A field declared with a type but no explicit field name is called an embedded field. An embedded field must be specified as a type name T or as a pointer to a non-interface type name *T, and T itself may not be a pointer type. The unqualified type name acts as the field name.

// A struct with four embedded fields of types T1, *T2, P.T3 and *P.T4
struct {
  T1        // field name is T1
  *T2       // field name is T2
  P.T3      // field name is T3
  *P.T4     // field name is T4
  x, y int  // field names are x and y
}

The following declaration is illegal because field names must be unique in a struct type:

struct {
  T     // conflicts with embedded field *T and *P.T
  *T    // conflicts with embedded field T and *P.T
  *P.T  // conflicts with embedded field T and *T
}

A struct is a collection of fields.

package main

import "fmt"

type Vertex struct {
  X int
  Y int
}

func main() {
  v := Vertex{10, 201}
  v.X = 4
  fmt.Println(v)
}

Output:

{4 201}

Struct in C

Before we will see the origin of everything in C, let's see the code written in C and the comments of how it would be in Golang. And we have it written in Golang, so that you can see how dynamic you were in Go, the C inheritance was simply fantastic and light.

#include <stdio.h>
#include <stdlib.h>

struct Login {
    char *User;
    char *Email;
};

/**
  // In Go would look like this
  type Login struct {
    User string
    Email string
  }
 */

int main() {
/** 
 // In Go 
func main() {
*/
    
    struct Login login, *pointerToLogin;

    /**
    // In Go
    login := Login{User:"jeffotoni", Email:"[email protected]"}
    */

    login.User = "jeffotoni";
    login.Email = "[email protected]";

    pointerToLogin = malloc(sizeof(struct Login));
    pointerToLogin->User = "pike";
    pointerToLogin->Email = "[email protected]";

    /**
    var pointerToLogin  = new(Login)
    pointerToLogin.User  = "pike"
    pointerToLogin.Email = "[email protected]"
    */
    printf("login vals: %s %s\n", login.User, login.Email);
    printf("pointerToLogin: %p %s %s\n", pointerToLogin, pointerToLogin->User, pointerToLogin->Email);

    /**
     fmt.Printf("login vals: %s %s\n", login.User, login.Email)
     fmt.Printf("pointerToLogin: %v %s %s\n", pointerToLogin, pointerToLogin.User, pointerToLogin.Email);
    */

    free(pointerToLogin);
    return 0;
}

Compile the program in C:

$ gcc -o struct-1-c struct-1-c.c

Output:

login vals: jeffotoni [email protected]
pointerToLogin: 0x5627788a0260 pike [email protected]

Confira agora o Código em Go abaixo:

package main

import "fmt"
// #include <stdio.h>
// #include <stdlib.h>

// struct Login {
//     char *User;
//     char *Email;
// };

//In Go would look like this
type Login struct {
    User  string
    Email string
}

// int main() {

// In Go
func main() {

    //struct Login login, *pointerToLogin;

    // In Go
    login := Login{User: "jeffotoni", Email: "[email protected]"}

    login.User = "jeffotoni"
    login.Email = "[email protected]"

    // pointerToLogin = malloc(sizeof(struct Login));
    // pointerToLogin->User = "pike";
    // pointerToLogin->Email = "[email protected]";

    var pointerToLogin = new(Login)
    pointerToLogin.User = "pike"
    pointerToLogin.Email = "[email protected]"

    //printf("login vals: %s %s\n", login.User, login.Email);
    //printf("pointerToLogin: %p %s %s\n", pointerToLogin, pointerToLogin->User, pointerToLogin->Email);

    fmt.Printf("login vals: %s %s\n", login.User, login.Email)
    fmt.Printf("pointerToLogin: %v %s %s\n", pointerToLogin, pointerToLogin.User, pointerToLogin.Email)

    //free(pointerToLogin);
    return
}

login vals: jeffotoni [email protected]
pointerToLogin: &{pike [email protected]} pike [email protected]

Struct Type Tags Json

We use json tags with omitempty or not to represent our json string, when generated by json.Marshal which we will see soon below.

Example:

import (
	//"encoding/json"
	"fmt"
)

// We use the omitempty tag in 'json: field, omitempty'
// when we want the field not to appear after
// making the marshal if it is empty.
type D struct {
	Height int
	Width  int `json:"width,omitempty"`
}

// Fields that do not have the "omitempty" tag will display 
// the same field being empty when generating 
// json through marshal.
type Cat struct {
	Breed    string `json:"breed,omitempty"`
	WeightKg int    `json:WeightKg`
	Size     D      `json:"size,omitempty"`
}

func main() {
	d := Cat{
		//Breed:    "Persa",
		WeightKg: 23,
		//Size:     D{20, 60},
	}
	//b, _ := json.Marshal(d)
	//fmt.Println(string(b))
	fmt.Println(d)
}

Output:

{ 23 {20 60}}

In this example we have a struct "ApiLogin" and inside it we have And1 *struct", ie, referencing a pointer from another struct, beautiful is not. To initialize and access the "And1" field, we have to use the "&" operator and create the struct at its initialization because it has not been defined yet so it would look like this: "And1: & struct {City string} {City: "BH"}"

type ApiLogin struct {
	And1  *struct {
		City string
	}
}

In this example we only time the pointer of a struct already defined above. To access it we do not need to create it again, just refer to it with "&" and this way "And2: &Address{}", very nice, right?

type ApiLogin struct {
	And2 *Address
}

The example below is a way to display various forms of initialization using struct.

package main

import "fmt"

type Address struct {
	City         string `json:"city"`
	Neighborhood string `json:"neighborhood"`
	Zipcode      string `json:"zipcode"`
}

type ApiLogin struct {
	Name  string `json:"name"`
	Cpf   string `json:"cpf"`
	Login string `json:"login"`
	Email string `json:"email"`

	// anonymous
	And1 *struct {
		City string
	}

	// pointer
	And2 *Address

	// list Address
	And3 []Address
}

func main() {

	// different ways to inizialize a struct
	//
	//

	apilogin1 := &ApiLogin{Name: "@jeffotoni", Cpf: "093.393.334-34", And1: &struct{ City string }{City: "BH"}}
	fmt.Println(apilogin1)
	fmt.Println(apilogin1.Name)
	fmt.Println(apilogin1.And1)
	fmt.Println(apilogin1.And1.City)

	apilogin2 := &ApiLogin{Name: "@jeffotoni", Cpf: "093.393.334-34", And2: &Address{City: "BH"}}
	fmt.Println(apilogin2)
	fmt.Println(apilogin2.Name)
	fmt.Println(apilogin2.And2)
	fmt.Println(apilogin2.And2.City)

	apilogin3 := &ApiLogin{Name: "@jeffotoni", Cpf: "093.393.334-34", And1: &struct{ City string }{City: "BH"}, And2: &Address{City: "BH"}}
	fmt.Println(apilogin3)
	fmt.Println(apilogin3.Name)
	fmt.Println(apilogin3.And1)
	fmt.Println(apilogin3.And1.City)
	fmt.Println(apilogin3.And2)
	fmt.Println(apilogin3.And2.City)

	var apilogin4 ApiLogin
	fmt.Println(apilogin4)

	apilogin5 := ApiLogin{}
	fmt.Println(apilogin5)

	apilogin6 := &ApiLogin{}
	fmt.Println(apilogin6)

	apilogin7 := new(ApiLogin)
	fmt.Println(apilogin7)

	// another way to feed the struct
	g1add := Address{City: "Belo Horizonte"}
	g2add := Address{City: "Curitiba"}

	// declaring as list
	gall := []Address{}

	// add items
	gall = append(gall, g1add)
	gall = append(gall, g2add)

	fmt.Println(gall)

	// initializes Struct
	apil3 := ApiLogin{}

	// recive same type
	apil3.And3 = gall

	// show struct
	fmt.Println(apil3)

	// another way to initialize and feed the struct list
	apil3.And3 = []Address{{City: "Sao Paulo"}, {City: "Brasilia"}}

	// show struct
	fmt.Println(apil3)
}

Output:

&{@jeffotoni 093.393.334-34   0xc00000e1e0 <nil> []}
@jeffotoni
&{BH}
BH
&{@jeffotoni 093.393.334-34   <nil> 0xc000060150 []}
@jeffotoni
&{BH  }
BH
&{@jeffotoni 093.393.334-34   0xc00000e300 0xc0000601b0 []}
@jeffotoni
&{BH}
BH
&{BH  }
BH
{    <nil> <nil> []}
{    <nil> <nil> []}
&{    <nil> <nil> []}
&{    <nil> <nil> []}
[{Belo Horizonte  } {Curitiba  }]
{    <nil> <nil> [{Belo Horizonte  } {Curitiba  }]}
{    <nil> <nil> [{Sao Paulo  } {Brasilia  }]}

The example below is to demonstrate the ease we have when we manipulate structs in Golang. We are initializing the struct with values and displaying on the screen.

Take a look:

package main

import "fmt"

type jsoninput []struct {
	Data string `json:"data"`
}

func main() {

	data := &jsoninput{{Data: "some data"}, {Data: "some more data"},
		{Data: "some more data"}}
	fmt.Println(data)

	// output:
	//  &[{some data} {some more data} {some more data}]
}

&[{some data} {some more data} {some more data}]

In our example below is another way to make array of struct, we made the statement at the time of initializing our struct, this causes that the struct does not get caught in only accepting array of struct. We did append in the fields and the magic happens.

Let's take a look at the complete code:

package main

import (
	"fmt"
)

type User struct {
	FirstName string `tag_name:"firstname"`
	LastName  string `tag_name:"lastname"`
	Age       int    `tag_name:"age"`
}

func main() {
	// create instance
	// slice struct
	users := []*User{}

	user := new(User)
	user.FirstName = "Jefferson"
	user.LastName = "otoni"
	user.Age = 350
	users = append(users, user)

	user = new(User)
	user.FirstName = "Pike"
	user.LastName = "Hob"
	user.Age = 55
	users = append(users, user)

	fmt.Println(users)

	for k, v := range users {
		fmt.Println(k, v)
		fmt.Println(v.FirstName)
		fmt.Println(v.LastName)
		fmt.Println(v.Age)
	}
}

Output:

[0xc000060150 0xc000060180]
0 &{Jefferson otoni 350}
Jefferson
otoni
350
1 &{Pike Hob 55}
Pike
Hob
55

Example Struct AWS Sqs Json

type SqsJson struct {
	Type      string `json:"type"`
	MessageId string `json:"messageid"`
	TopicArn  string `json:"topicarn"`
	Message   string `json:"message"`
	//Message          JsonMessage
	Timestamp        string `json:"timestamp"`
	SignatureVersion string `json:"signatureversion"`
	Signature        string `json:"signature"`
	SigningCertURL   string `json:"signingcerturl"`
	UnsubscribeURL   string `json:"unsubscribeurl"`
}

type JsonMessage struct {
	NotificationType string `json:"notificationType"`
	Bounce           struct {
		BounceType        string `json:"bounceType"`
		BounceSubType     string `json:"bounceSubType"`
		BouncedRecipients []struct {
			EmailAddress   string `json:"emailAddress"`
			Action         string `json:"action"`
			Status         string `json:"status"`
			DiagnosticCode string `json:"diagnosticCode"`
		} `json:"bouncedRecipients"`
		Timestamp    time.Time `json:"timestamp"`
		FeedbackID   string    `json:"feedbackId"`
		RemoteMtaIP  string    `json:"remoteMtaIp"`
		ReportingMTA string    `json:"reportingMTA"`
	} `json:"bounce"`

	Mail struct {
		Timestamp        time.Time `json:"timestamp"`
		Source           string    `json:"source"`
		SourceArn        string    `json:"sourceArn"`
		SourceIP         string    `json:"sourceIp"`
		SendingAccountID string    `json:"sendingAccountId"`
		MessageID        string    `json:"messageId"`
		Destination      []string  `json:"destination"`
		HeadersTruncated bool      `json:"headersTruncated"`
		Headers          []struct {
			Name  string `json:"name"`
			Value string `json:"value"`
		} `json:"headers"`

		CommonHeaders struct {
			From    []string `json:"from"`
			ReplyTo []string `json:"replyTo"`
			To      []string `json:"to"`
			Subject string   `json:"subject"`
		} `json:"commonHeaders"`
	} `json:"mail"`
}

When the subject is struct we have several possibilities to deal with and work with this feature in Golang, it is practically embedded in everything we will build in Golang, Structs is something powerful in Go to manipulate data and send data all the time between channels using goroutine, be it in queues, database recordings, json and database reads, GraphQL, REST API, SOAP etc ...

Fatih Structs to Map

Below is a lib that works directly with struct, converting to Maps. I do not use it in production but for our course it is interesting to analyze. We know that the more native we are in Golang it will be a good option, but at times we will need some libs to help us. This project is not maintained anymore and is archived. Feel free to fork and make your own changes if needed.

Structs contains various utilities to work with Go (Golang) structs. It was initially used by me to convert a struct into a map[string]interface{}. With time I've added other utilities for structs. It's basically a high level package based on primitives from the reflect package. Feel free to add new functions or improve the existing code.

Install

go get github.com/fatih/structs

To test low rotate the code below:

type Server struct {
	Name        string `json:"name,omitempty"`
	ID          int
	Enabled     bool
	Users       []string // not exported
	http.Server          // embedded
}

func main() {

	// Create a new struct type:

	server := &Server{
		Name:    "gopher",
		ID:      12345678,
		Users:   []string{"jeffotoni", "pike", "dennis", "ken"},
		Enabled: true,
	}

	// struct
	fmt.Println(server)

	// create struct fatih
	s := structs.New(server)
	m := s.Map() // Get a map[string]interface{}
	fmt.Println(m)

	v := s.Values() // Get a []interface{}
	fmt.Println(v)

	f := s.Fields() // Get a []*Field
	fmt.Println(f)

	n := s.Names() // Get a []string
	fmt.Println(n)

	name := s.Field("Name") // Get a *Field based on the given field name

	// Get the underlying value,  value => "gopher"
	value := name.Value().(string)
	fmt.Println(value)

	tagValue := name.Tag("json")
	fmt.Println(tagValue)

	f1, ok := s.FieldOk("Name") // Get a *Field based on the given field name
	fmt.Println(f1, ok)

	n2 := s.Name() // Get the struct name
	fmt.Println(n2)

	h := s.HasZero() // Check if any field is uninitialized
	fmt.Println(h)
}

Output:

&{gopher 12345678 true [jeffotoni pike dennis ken] 
{ <nil> <nil> 0s 0s 0s 0s 0 map[] <nil> <nil> 0 0 {{0 0} 0} <nil> {0 0} map[] map[] <nil> []}}
map[Server:map[TLSConfig:<nil> ReadHeaderTimeout:0s IdleTimeout:0s Addr: Handler:<nil> 
]MaxHeaderBytes:0 TLSNextProto:map[] ConnState:<nil> ErrorLog:<nil> ReadTimeout:0s WriteTimeout:0s] 
Name:gopher ID:12345678 Enabled:true Users:[jeffotoni pike dennis ken]]
[gopher 12345678 true [jeffotoni pike dennis ken]  <nil> <nil> 0s 0s 0s 0s 0 map[] <nil> <nil>]
[0xc000106000 0xc000106090 0xc000106120 0xc0001061b0 0xc000106240]
[Name ID Enabled Users Server]
gopher
name,omitempty
&{{0x626180 0xc0000f6000 408} {Name  0x626180 json:"name,omitempty" 0 [0] false} structs} true
Server
true

Map Types

A map is an unordered group of elements of one type, called the element type, indexed by a set of unique keys of another type, called the key type. The value of an uninitialized map is nil.

MapType     = "map" "[" KeyType "]" ElementType .
KeyType     = Type .

The comparison operators == and != must be fully defined for operands of the key type; thus the key type must not be a function, map, or slice. If the key type is an interface type, these comparison operators must be defined for the dynamic key values; failure will cause a run-time panic.

map[string]int
map[*T]struct{ x, y float64 }
map[string]interface{}

The number of map elements is called its length. For a map m, it can be discovered using the built-in function len and may change during execution. Elements may be added during execution using assignments and retrieved with index expressions; they may be removed with the delete built-in function.

A new, empty map value is made using the built-in function make, which takes the map type and an optional capacity hint as arguments:

make(map[string]int)
make(map[string]int, 100)

The initial capacity does not bound its size: maps grow to accommodate the number of items stored in them, with the exception of nil maps. A nil map is equivalent to an empty map except that no elements may be added.

Some examples of map initialization:

package main

import "fmt"

type linkResult struct {
	body string
	urls []string
}

type linkFetcher map[string]*linkResult

func main() {
	// Required to initialize
	// the map with values
	var m1 map[string]int
	var m2 = make(map[string]int)
	var m3 = map[string]int{"population": 500000}
	var m4 = m3
	var m5 map[string]string
	/* create a map*/
	m5 = make(map[string]string)
	fmt.Println(m1, m2, m3, m4, m5)

	var l = linkFetcher{
		"https://golang.org/": &linkResult{
			"The Go Programming Language",
			[]string{
				"https://golang.org/pkg/",
				"https://golang.org/cmd/",
			},
		},
		"https://golang.org/pkg/": &linkResult{
			"Packages",
			[]string{
				"https://golang.org/",
				"https://golang.org/cmd/",
				"https://golang.org/pkg/fmt/",
				"https://golang.org/pkg/os/",
			},
		}}

	fmt.Println(l)
}

Output:

map[] map[] map[population:500000] map[population:500000] map[]

A map is declared using the following syntax:

var m map[KeyType]ValueType

// For example, Here is how you can declare a map of string keys to int values
var m map[string]int

The zero value of a map is nil. A nil map has no keys. Moreover, any attempt to add keys to a nil map will result in a runtime error.

Let’s see an example:

package main

import "fmt"

func main() {

	// Required to initialize
	// the map with values
	var m map[string]int
	fmt.Println(m)
	if m == nil {
		fmt.Println("is nil")
	}
	// Attempting to add keys
	// to a nil map will result in a runtime error
	//m["population"] = 500000
	//fmt.Println(m)
}

Output:

map[]
is nil

You can initialize a map using the built-in make() function. You just need to pass the type of the map to the make() function as in the example below. The function will return an initialized and ready to use map.

// Initializing a map using the built-in make() function
var m = make(map[string]int)

Example:

package main

import "fmt"

func main() {

	// Required to initialize
	// the map with values
	var m = make(map[string]int)
	fmt.Println(m)
	if m == nil {
		fmt.Println("is nil")
	}
	m["population"] = 500000
	fmt.Println(m)
}

Output:

map[]
map[population:500000]

The example below introduces the creation of a map getting a struct done. When do we use an empty struct? There are some scenarios when we have a large amount of access in our API, but when I say large is> 10k = 10k of requests per second, in this scenario when we do our handler, we can implement a channel receiving an empty struct {} so that we can get put on a channel and process everything in with more security. We will show more ahead this very legal approach.

Example:

package main

import "fmt"

func main() {
	s := "key"
	seen := make(map[string]struct{}) // set of strings
	// ...
	if _, ok := seen[s]; !ok {
		seen[s] = struct{}{}
		// ...first time seeing s...
	}
	fmt.Println(seen)
}

Output:

map[key:{}]

we can also make our map key receive an empty struct. Well we know that it gets any type ie the struct can be complete without it's done too.

Example:

type T struct{}

func main() {
	s := T{}
	seen := make(map[struct{}]struct{}) // set of strings
	// ...
	if _, ok := seen[s]; !ok {
		seen[s] = struct{}{}
		// ...first time seeing s...
	}
	fmt.Println(seen)
}

Output:

map[{}:{}]

Map Literals Continued

A map literal is a very convenient way to initialize a map with some data. You just need to pass the key-value pairs separated by colon inside curly braces.

package main

import "fmt"

func main() {
	var m = map[string]string{
		"Brasil": "Brasilia",
		"EUA":    "Washington, D.c",
		"Italy":  "Roma",
		"France": "Paris",
		"Japan":  "Toquio",
	}

	fmt.Println(m)
}

Output:

map[Italy:Roma France:Paris Japan:Toquio Brasil:Brasilia EUA:Washington, D.c]

So you can check for the existence of a key in a map by using the following two-value assignment. The boolean variable ok will be true if the key exists, and false otherwise.

value, ok := m[key]

Consider the following map for example:

var C = map[string]string{
		"Brasil": "Brasilia",
		"EUA":    "Washington, D.c",
		"Italy":  "Roma",
		"France": "Paris",
		"Japan":  "Toquio",
	}

capital, ok := C["EUA"]  // "Washington, D.c", true

However, If you try to access a key that doesn’t exist, then the map will return an empty string "" (zero value of strings), and false

capital, ok := C["África do Sul"]  // "", false

You can delete a key from a map using the built-in delete() function. The syntax looks like this.

// Delete the `key` from the `map`
delete(map, key)

Example:

package main

import "fmt"

func main() {
	var country = map[string]string{
		"Brasil": "Brasilia",
		"EUA":    "Washington, D.c",
		"Italy":  "Roma",
		"France": "Paris",
		"Japan":  "Toquio",
	}
	delete(country, "Japan")
	delete(country, "Italy")
	fmt.Println(country)
}

Output:

map[Brasil:Brasilia EUA:Washington, D.c France:Paris]

If the top-level type is just a type name, you can omit it from the elements of the literal.

package main

import "fmt"

type Login struct {
	User, Login, Email string
}

// passing a struct as parameter
// for our struct map
var m = map[string]Login{
	"jeffotoni": {"jeffotoni", "jeff", "[email protected]"},
	"Google":    {"root", "super", "[email protected]"},
}

func main() {
	fmt.Println(m)
}
```go

```bash
map[jeffotoni:{jeffotoni jeff [email protected]} Google:{root super [email protected]}]

Channel Types

A channel provides a mechanism for concurrently executing functions to communicate by sending and receiving values of a specified element type. The value of an uninitialized channel is nil.

ChannelType = ( "chan" | "chan" "<-" | "<-" "chan" ) ElementType .

The optional <- operator specifies the channel direction, send or receive. If no direction is given, the channel is bidirectional. A channel may be constrained only to send or only to receive by conversion or assignment.

chan T          // can be used to send and receive values of type T
chan<- float64  // can only be used to send float64s
<-chan int      // can only be used to receive ints

The <- operator associates with the leftmost chan possible:

chan<- chan int    // same as chan<- (chan int)
chan<- <-chan int  // same as chan<- (<-chan int)
<-chan <-chan int  // same as <-chan (<-chan int)
chan (<-chan int)

A new, initialized channel value can be made using the built-in function make, which takes the channel type and an optional capacity as arguments:

make(chan int, 100)

The capacity, in number of elements, sets the size of the buffer in the channel. If the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready. Otherwise, the channel is buffered and communication succeeds without blocking if the buffer is not full (sends) or not empty (receives). A nil channel is never ready for communication.

A channel may be closed with the built-in function close. The multi-valued assignment form of the receive operator reports whether a received value was sent before the channel was closed.

A single channel may be used in send statements, receive operations, and calls to the built-in functions cap and len by any number of goroutines without further synchronization. Channels act as first-in-first-out queues. For example, if one goroutine sends values on a channel and a second goroutine receives them, the values are received in the order sent.

Let me show you an example:

package main

import (
  "fmt"
  "os"
  "time"
)

type Promise struct {
  Result chan string
  Error  chan error
}

var (
  ch1  = make(chan *Promise)  // received a pointer from the structure
  ch2  = make(chan string, 1) // allows only 1 channels
  ch3  = make(chan int, 2)    // allows only 2 channels
  ch4  = make(chan float64)   // has not been set can freely receive
  ch5  = make(chan []byte)    // by default the capacity is 0
  ch6  = make(chan bool, 1)   // non-zero capacity
  ch7  = make(chan time.Time, 2)
  ch8  = make(chan struct{}, 2)
  ch9  = make(chan struct{})
  ch10 = make(map[string](chan int)) // map channel
  ch11 = make(chan error)
  ch12 = make(chan error, 2)
  // receives a zero struct
  ch14 <-chan struct{}
  ch15 = make(<-chan bool)          // can only read from
  ch16 = make(chan<- []os.FileInfo) // // can only write to
  // holds another channel as its value
  ch17 = make(chan<- chan bool) // // can read and write to
)

// Parameters of Func
// (jobs <-chan int, results chan<- int)

// Receives Value, only read
// jobs <-chan int //receives the value

// Receives Channel, only write
// results chan<- int // receive channel
// or
// results chan int // receive channel

// Receives Channel variadic
// results ...<-chan int

func main() {

  ch2 <- "okay"
  defer close(ch2)
  fmt.Println(ch2, &ch2, <-ch2)

  ch7 <- time.Now()
  ch7 <- time.Now()
  fmt.Println(ch7, &ch7, <-ch7)
  fmt.Println(ch7, &ch7, <-ch7)
  defer close(ch7)

  ch3 <- 1 // okay
  ch3 <- 2 // okay
  // deadlock
  // ch3 <- 3 // does not accept any more values, if you do it will error : deadlock
  defer close(ch3)
  fmt.Println(ch3, &ch3, <-ch3)
  fmt.Println(ch3, &ch3, <-ch3)

  ch10["lambda"] = make(chan int, 2)
  ch10["lambda"] <- 100
  defer close(ch10["lambda"])
  fmt.Println(<-ch10["lambda"])
}

Output:

0xc000056180 0x55bb00 okay
0xc0000561e0 0x55bb28 2019-01-25 15:11:41.982906669 -0200 -02 m=+0.000147197
0xc0000561e0 0x55bb28 2019-01-25 15:11:41.982906922 -0200 -02 m=+0.000147409
0xc00001e0e0 0x55bb08 1
0xc00001e0e0 0x55bb08 2
100

Blank Identifier

The blank identifier is represented by the underscore character _. It serves as an anonymous placeholder instead of a regular (non-blank) identifier and has special meaning in declarations, as an operand, and in assignments.

Example:

// function statement
func f() (int, string, error)

// function return
_, _, _ := f()

Interface Types

An interface is two things:

it is a set of methods
but it is also a type

The interface{} type, the empty interface is the interface that has no methods

Since there is no implements keyword, all types implement at least zero methods, and satisfying an interface is done automatically, all types satisfy the empty interface. That means that if you write a function that takes an interface{} value as a parameter, you can supply that function with any value.

Example:

func DoSomething(v interface{}) {
   // ...
}

var Msg interface{}

type Stringer interface {
    String() string
}

Here's an interface as a method

An interface type specifies a method set called its interface. A variable of interface type can store a value of any type with a method set that is any superset of the interface. Such a type is said to implement the interface. The value of an uninitialized variable of interface type is nil.

InterfaceType      = "interface" "{" { MethodSpec ";" } "}" .
MethodSpec         = MethodName Signature | InterfaceTypeName .
MethodName         = identifier .
InterfaceTypeName  = TypeName .

As with all method sets, in an interface type, each method must have a unique non-blank name.

// A simple File interface
interface {
  Read(b Buffer) bool
  Write(b Buffer) bool
  Close()
}

More than one type may implement an interface. For instance, if two types S1 and S2 have the method set

func (p T) Read(b Buffer) bool { return … }
func (p T) Write(b Buffer) bool { return … }
func (p T) Close() { … }

(where T stands for either S1 or S2) then the File interface is implemented by both S1 and S2, regardless of what other methods S1 and S2 may have or share.

A type implements any interface comprising any subset of its methods and may therefore implement several distinct interfaces. For instance, all types implement the empty interface:

interface{}

Similarly, consider this interface specification, which appears within a type declaration to define an interface called Locker:

type Locker interface {
  Lock()
  Unlock()
}

If S1 and S2 also implement

func (p T) Lock() { … }
func (p T) Unlock() { … }

they implement the Locker interface as well as the File interface.

An interface T may use a (possibly qualified) interface type name E in place of a method specification. This is called embedding interface E in T; it adds all (exported and non-exported) methods of E to the interface T.

type ReadWriter interface {
  Read(b Buffer) bool
  Write(b Buffer) bool
}

type File interface {
  ReadWriter  // same as adding the methods of ReadWriter
  Locker      // same as adding the methods of Locker
  Close()
}

type LockedFile interface {
  Locker
  File        // illegal: Lock, Unlock not unique
  Lock()      // illegal: Lock not unique
}

An interface type T may not embed itself or any interface type that embeds T, recursively.

// illegal: Bad cannot embed itself
type Bad interface {
  Bad
}

// illegal: Bad1 cannot embed itself using Bad2
type Bad1 interface {
  Bad2
}
type Bad2 interface {
  Bad1
}

Example:

package main

import (
  "fmt"
)

type R struct {
  R string
}

type Iread interface {
  Read() string
}

func (r *R) Read() string {
  return fmt.Sprintf("Only: call Read")
}

func Call(ir Iread) string {
  return fmt.Sprintf("Read: %s", ir.Read())
}

func main() {
  var iread Iread
  r := R{"hello interface"}
  // A way to use Interface
  iread = &r
  fmt.Println(iread, r)
  fmt.Println(iread.Read())

  // Second way to access interface
  r2 := R{"hello interface call"}
  fmt.Println(Call(&r2))
}

Output:

&{hello interface} {hello interface}
Only: call Read
Read: Only: call Read

Interface as type

Interfaces as type interface{} means you can put value of any type, including your own custom type. All types in Go satisfy an empty interface (interface{} is an empty interface). In your example, Msg field can have value of any type.

var val interface{} // element type of m is assignable to val

type Empty interface {
    /* it has no methods */
}

// Because, Empty interface has no methods, 
// following types satisfy the Empty interface
var a Empty

a = 60
a = 10.5
a = "Lambda Man"

Interfaces as types looks at another example below:

package main

import (
  "fmt"
)

type MyStruct struct {
  Msg interface{}
}

func main() {
  b := MyStruct{}
  // string
  b.Msg = "5"
  fmt.Printf("%#v %T \n", b.Msg, b.Msg) // Output: "5" string

  // int
  b.Msg = 5
  fmt.Printf("%#v %T", b.Msg, b.Msg) //Output:  5 int

  // map
  b.Msg = map[string]string{"population": "500000", "language": "sueco"}
  fmt.Printf("%#v %T", b.Msg, b.Msg) //Output:  5 int
}

Exercise One

Exercise: Fill in the struct JsonMessage AWS above, initialize the struct and fill in the fields, and make a fmt.Println to display the filled fields. To be more readable you can separate into each struct type struct.

Control Structures

Control

The control structures are:

For, If, else, else if

And some statments between them: break, continue, switch, case and goto.

Control Return

Statements control execution.

Statement =
  Declaration | LabeledStmt | SimpleStmt |
  GoStmt | ReturnStmt | BreakStmt | ContinueStmt | GotoStmt |
  FallthroughStmt | Block | IfStmt | SwitchStmt | SelectStmt | ForStmt |
  DeferStmt .

SimpleStmt = EmptyStmt | ExpressionStmt | SendStmt | IncDecStmt | Assignment | ShortVarDecl .

A terminating statement prevents execution of all statements that lexically appear after it in the same block. The following statements are terminating:

A "return" or "goto" statement.

Return:

package main

func main() {
  println(Lambda())
  return
}

func Lambda() string {

  return "Lambda"
}

Output:

Lambda

Control Goto

Goto:

package main

import "fmt"

func main() {
  n := 0

LOOP1:
  n++
  if n == 10 {
    println("fim")
    return
  }

  if n%2 == 0 {
    goto LOOP2
  } else {

    fmt.Println("n", n, "LOOP1 here...")
    goto LOOP1
  }

LOOP2:
  fmt.Println("n", n, "LOOP2 here...")
  goto LOOP1

}

Output:

n 1 LOOP1 here...
n 2 LOOP2 here...
n 3 LOOP1 here...
n 4 LOOP2 here...
n 5 LOOP1 here...
n 6 LOOP2 here...
n 7 LOOP1 here...
n 8 LOOP2 here...
n 9 LOOP1 here...
fim

Control if Else

An "if" statement in which:
- the "else" branch is present, and
- both branches are terminating statements.

package main

func main() {
  n := 100
  if n > 0 && n <= 55 {
    println("n > 0 or n <= 55")
  } else if n > 56 && n < 70 {
    println("n > 56 and n < 70")
  } else {

    if n >= 100 {
      println(" else here.. n > 100")
    } else {
      println(" else here.. n > 70")
    }
  }
}

Output:

else here.. n > 100

Control For Break Continue

A "for" statement in which:
- there are no "break" statements referring to the "for" statement, and
- the loop condition is absent.
- there are "continue"
- A "break" statement terminates execution of the innermost "for", "switch", or "select" statement within the same

package main

func main() {
  // will be looping infinitely
  // for {
  // }

  // will run only once and exit
  for {
    break
  }

  n := 5
  for n > 0 {
    n--
    println(n)
  }
  // Output:
  // 4
  // 3
  // 2
  // 1
  // 0

  // declaring i no and increasing i
  for i := 0; i < 5; i++ {
    println(i)
  }
  // Output:
  // 0
  // 1
  // 2
  // 3
  // 4

  n = 5
  for i := 0; i < n; i++ {
    if i <= 2 {
      continue
    } else {
      println("i > 2 = ", i)
    }
  }

  // Output:
  // i > 2 =  3
  // i > 2 =  4

  n = 5
  for i := 0; i < n; i++ {
    if i == 2 {
      break
    } else {
      println("i: ", i)
    }
  }
  // Output:
  // i:  0
  // i:  1
  
  // infinitely
  for ; ; i++ {
    println("i: ", i)
  }
  // Output:
  // i:  1
  // i:  2
  // ..
  // ..
}

Control Switch Case Break

A "switch" statement in which:
- there are no "break" statements referring to the "switch" statement,
- there is a default "case", and
- the statement lists in each case, including the default, end in a terminating statement, or a possibly labeled "fallthrough" statement.

package main

func main() {
  j := 10
  i := 0
  switch j {
  case 11:
    println("here: 11")
    break
  default:
    println("here default")
    break
  }

  // infinitely
  for ; ; i++ {

    switch i {
    case 5:
      goto LABELS
    case i:
      println("i: ", i)
      break
    default:
      println("default: ", i)
    }
  }

LABELS:
  f()

}

func f() {
  println("goto fim")
}

Output:

here default
i:  0
i:  1
i:  2
i:  3
i:  4
goto fim

Control Label

A labeled statement labeling a terminating statement.

package main

func main() {
  i := 0
  // infinitely
  for ; ; i++ {
    for {
      if i == 10 {
        goto LABEL
      }
      i++
    }
  }

LABEL:
  f(i)

}

func f(i int) {
  println("label fim i:", i)
}

Output:

label fim i: 10

All other statements are not terminating.

A statement list ends in a terminating statement if the list is not empty and its final non-empty statement is terminating.

A "for" statement with a "range" clause iterates through all entries of an array, slice, string or map, or values received on a channel. For each entry it assigns iteration values to corresponding iteration variables if present and then executes the block.

RangeClause = [ ExpressionList "=" | IdentifierList ":=" ] "range" Expression .

Control Range

The expression on the right in the "range" clause is called the range expression, which may be an array, pointer to an array, slice, string, map, or channel permitting receive operations. As with an assignment, if present the operands on the left must be addressable or map index expressions; they denote the iteration variables. If the range expression is a channel, at most one iteration variable is permitted, otherwise there may be up to two. If the last iteration variable is the blank identifier, the range clause is equivalent to the same clause without that identifier.

Range expression                          1st value          2nd value

array or slice  a  [n]E, *[n]E, or []E    index    i  int    a[i]       E
string          s  string type            index    i  int    see below  rune
map             m  map[K]V                key      k  K      m[k]       V
channel         c  chan E, <-chan E       element  e  E

See an example below, with various uses using Range:

package main

import "fmt"

func main() {

  // string arrays / slice
  var lang = [...]string{"Erlang", "Elixir", "Haskell", "Clojure", "Scala"}

  // screen list
  fmt.Println(lang)

  // list the positions srtring arrays
  for k, v := range lang {
    fmt.Println(k, v)
  }

  /* create a map*/
  countryCapitalMap := map[string]string{"Brasil": "Brasilia", "EUA AMERICA": "Washington, D.C.", "France": "Paris", "Italy": "Roma", "Japan": "Tokyo"}

  /* print map using key-value*/
  for country, capital := range countryCapitalMap {
    fmt.Println("Capital of", country, "is", capital)
  }

  // channel
  jobs := make(chan int, 3)

  // for channel
  for j := 1; j <= 3; j++ {
    jobs <- j
  }
  // println(<-jobs)
  // println(<-jobs)
  // println(<-jobs)

  // close
  /* date is required for range to work*/
  close(jobs)

  /* This syntax is valid too. */
  for range jobs {
  }

  /* it is mandatory to close the channels to be able to scroll */
  for ch := range jobs {
    println(ch)
  }

  // it is not an array struct, it will range from error.
  sa := struct{ nick string }{"@jeffotoni"}
  fmt.Println(sa.nick)

  // here the range will be able to list all struct
  a := []struct{ nick string }{{"@devopsbr"}, {"@go_br"}, {"@awsbrasil"}, {"@go_br"}, {"@devopsbh"}}
  for i, v := range a {
    fmt.Println(i, v.nick)
  }

  // struct pointer
  var testdata *struct {
    a *[3]int
  }
  for i := range testdata.a {
    // testdata.a is never evaluated; len(testdata.a) is constant
    // i ranges from 0 to 2
    fmt.Println(i)
  }

  // new example interface and range
  var key string
  var val interface{} // element type of m is assignable to val
  m := map[string]int{"mon": 0, "tue": 1, "wed": 2, "thu": 3, "fri": 4, "sat": 5, "sun": 6}
  for key, val = range m {
    fmt.Println(key, val)
  }
}

Output:

[Erlang Elixir Haskell Clojure Scala]
0 Erlang
1 Elixir
2 Haskell
3 Clojure
4 Scala
Capital of Brasil is Brasilia
Capital of EUA AMERICA is Washington, D.C.
Capital of France is Paris
Capital of Italy is Roma
Capital of Japan is Tokyo
@jeffotoni
0 @devopsbr
1 @go_br
2 @awsbrasil
3 @go_br
4 @devopsbh
0
1
2
sat 5
sun 6
mon 0
tue 1
wed 2
thu 3
fri 4

Errors

The predeclared type error is defined as

type error interface {
   Error() string
}

It is the conventional interface for representing an error condition, with the nil value representing no error. For instance, a function to read data from a file might be defined:

func Read(f *File, b []byte) (n int, err error)

Introduction Errors

Error handling in Golang is very simple to deal with. There is no try, catch or exceptions. The error is handled with every call of some function. As we show "error" is an interface, and much used.

When we talk about handling errors in Golang everything is very simple, a good practice is to return an error in the functions that we created and treat them.

Let's see in practice how it works.

package main

import "fmt"

func main() {
	var error error
	fmt.Println(error)
}

Output:

<nil>

How Error Control Works

Example:

package main

import (
	"encoding/json"
	"fmt"
)

var Error error
var b []byte

func main() {
	fmt.Println(Error)
	cpu := make(chan int)
	// Create JSON from the instance data.
	b, Error = json.Marshal(cpu)
	if Error != nil {
		fmt.Println(Error)
	} else {
		fmt.Println(string(b))
	}
}

Output:

<nil>
json: unsupported type: chan int

Errors New

package main

import (
	"errors"
	"fmt"
	"math"
)

func Sqrt(fvalue float64) (float64, error) {
	if fvalue < 0 {
		return 0, errors.New("Math: negative number passed to Sqrt [" + fmt.Sprintf("%.2f", fvalue) + "]")
	}
	return math.Sqrt(fvalue), nil
}

func main() {
	result, err := Sqrt(-33)

	if err != nil {
		fmt.Println(err)
	} else {
		fmt.Println("Sqrt (-33) = [", result, "]")
	}

	result, err = Sqrt(81)

	if err != nil {
		fmt.Println(err)
	} else {
		fmt.Println("Sqrt(81) = [", result, "]")
	}
}

Output:

Math: negative number passed to Sqrt [-33.00]
Sqrt(81) = [ 9 ]

Custom Errors

package main

import "fmt"

// It's possible to use custom types as `error`s by
// implementing the `Error()` method on them.
type MyError struct {
    line int
    msg  string
}

func (e *MyError) Error() string {
    return fmt.Sprintf("%d - %s", e.line, e.msg)
}

func MyFunc(line int) (int, error) {
    if line < 100 {

        // In this case we use `&MyError` syntax to build
        // a new struct, supplying values for the two
        // fields `arg` and `prob`.
        return -1, &MyError{line, "can't work with it"}
    }
    return line, nil
}

func main() {

    // The one loops below test out each of our
    // error-returning functions.
    for _, i := range []int{200, 99} {
        if r, e := MyFunc(i); e != nil {
            fmt.Println("MyFunc failed:", e)
        } else {
            fmt.Println("MyFunc worked in line: ", r)
        }
    }
}

Output:

MyFunc worked in line:  200
MyFunc failed: 99 - can't work with it

fmt Errorf

package main

import (
    "fmt"
    "math"
)

func circleArea(radius float64) (float64, error) {
    if radius < 0 {
        return 0, fmt.Errorf("Failed, radius %0.2f is less than zero", radius)
    }
    return math.Pi * radius * radius, nil
}

func main() {
    radius := -80.0
    area, err := circleArea(radius)
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Printf("Area of circle: %0.2f", area)
}

Output:

Area calculation failed, radius -80.00 is less than zero

Functions

Declaring and Calling Functions in Golang. In Golang, we declare a function using the func keyword. A function has a name, a list of comma-separated input parameters along with their types, the result type(s), and a body. The input parameters and return type(s) are optional for a function.

Example of declaring and Calling Functions in Golang:

func Sum(x float64, y float64) float64 {
	return (x + y) / 2
}

Introduction Function

Go requires explicit returns, i.e. it won’t automatically return the value of the last expression. When you have multiple consecutive parameters of the same type, you may omit the type name for the like-typed parameters up to the final parameter that declares the type. A function type denotes the set of all functions with the same parameter and result types. The value of an uninitialized variable of function type is nil.

Some possibilities:

func()
func(x int) int
func(a, _ int, z float32) bool
func(a, b int, z float32) (bool)
func(prefix string, values ...int)
func(a, b int, z float64, opt ...interface{}) (success bool)
func(int, int, float64) (float64, *[]int)
func(n int) func(p *T)

package main

func F1(name string) string {
    return "Hello, " + name
}

func main() {

    println(F1("@go_br"))
}

Output:

Hello, @go_br

Return Multiple Values

Go has built-in support for multiple return values. This feature is used often in idiomatic Go, for example to return both result and error values from a function.

package main

import "fmt"

// The `(int, int)` in this function signature shows that
// the function returns 2 `int`s.
func F1() (string, int, error) {
    return "@go_br", 100, nil
}

func main() {

    // Here we use the 2 different return values from the
    // call with _multiple assignment_.
    a, b, err := F1()
    fmt.Println(a)
    fmt.Println(b)
    fmt.Println(err)

    // If you only want a subset of the returned values,
    // use the blank identifier `_`.
    a, _, err = F1()
    fmt.Println(a)
    fmt.Println(err)
}

Output:

@go_br
100
<nil>
@go_br
<nil>

Variadic Functions

Variadic functions can be called with any number of trailing arguments. For example, fmt.Println is a common variadic function. Here’s a function that will take an arbitrary number of ints as arguments. Variadic functions can be called in the usual way with individual arguments.

package main

import (
	"fmt"
	"strings"
)

// This variadic function takes an arbitrary number of ints as arguments.
func Show(names ...string) {
	fmt.Print("The Len of ", len(names)) // Also a variadic function.
	appends := ""
	for _, name := range names {
		appends += name + ","
	}
	appends = strings.Trim(appends, ",")
	fmt.Println(" is: [", appends, "]") // Also a variadic function.
}

func main() {

	// Variadic functions can be called in the usual way with individual
	// arguments.
	Show("C", "C++")
	Show("Clojure", "Elixir", "Scala")

	// If you already have multiple args in a slice, apply them to a variadic
	// function using func(slice...) like this.
	nums := []string{"Algol", "C", "C++", "Golang"}
	Show(nums...)
}

Output:

The Len of 2 is: [ C,C++ ]
The Len of 3 is: [ Clojure,Elixir,Scala ]
The Len of 4 is: [ Algol,C,C++,Golang ]

Functions as a Parameter

You can pass function as parameter to a Go function. Here is an example of passing function as parameter to another Go function.

package main

import "fmt"

// convert types take an int
// and return a string value.
type fn func(int) string

func f1(param int) string {
	return fmt.Sprintf("param is %v", param)
}

func f2(param int) string {
	return fmt.Sprintf("param is %v", param)
}

func test(f fn, val int) {
	fmt.Println(f(val))
}

func main() {
	test(f1, 432)
	test(f2, 874)
}

Output:

param is 432
param is 874

package main

import "fmt"

// --------------------------------
func Square(num int) int {
	return num * num
}

func Mapp(f func(int) int, List []int) []int {
	var a = make([]int, len(List), len(List))
	for index, val := range List {

		a[index] = f(val)
	}
	return a
}

func main() {
	list := []int{454, 455, 86, 988}
	result := Mapp(Square, list)
	fmt.Println(result)
}

Output:

[206116 207025 7396 976144]

Closures

Go supports anonymous functions, which can form closures. Anonymous functions are useful when you want to define a function inline without having to name it. This function intSeq returns another function, which we define anonymously in the body of intSeq. The returned function closes over the variable i to form a closure.

Example:

package main

import "fmt"

func PlusX() func(v int) int {
    return func(v int) int {
        return v + 5
    }
}

func plusXandY(x int) func(v int) int {
    return func(v int) int {
        return v + x
    }
}

func main() {
    p := PlusX()
    fmt.Printf("5+15: %d\n", p(15))

    px := plusXandY(6)
    fmt.Printf("6+10: %d\n", px(10))
}

Output:

5+15: 20
6+10: 16

Recursion

Go supports recursive functions. Here’s a classic factorial example.

A simple example:

package main

import "fmt"

func fact(n int) int {
	if n == 0 {
		return 1
	}
	return n * fact(n-1)
}

func main() {
	fmt.Println(fact(7))
}

Output:

Listing all subdirectories directories:

package main

import (
	"fmt"
	"os"
	"path"
	"path/filepath"
)

// Listing our example directory
// Listing our example directory recursively
func main() {

	// Capturing our path that is in the environment
	gopath := os.Getenv("PWD")

	// directory we want to list
	gopath += "/examples"

	// Making call in function
	list := ListDir(gopath)

	// listing the function return
	for i, p := range list {
		fmt.Printf("[%d:%s===%s]\n", i, path.Dir(p), path.Base(p))
	}
}

// This function uses pkg filepath.Walk, it is
// a recursive function, where it will go through
// our directory and its subfolders.
func ListDir(rootpath string) []string {

	list := make([]string, 0)

	// recursive call
	// This function receives a function as parameter and after going through all levels it ends.
	err := filepath.Walk(rootpath, func(path string, info os.FileInfo, err error) error {
		if info.IsDir() {
			return nil
		}

		if filepath.Ext(path) != ".git" && filepath.Ext(path) != ".svn" {
			list = append(list, path)
		}
		return nil
	})
	if err != nil {
		fmt.Printf("walk error [%v]\n", err)
	}
	return list
}

Output:

[0:$(pwd)/examples/bufio.writer===main.go]
[1:$(pwd)/examples/error===error1.go]
[2:$(pwd)/examples/error===error2.go]
[3:$(pwd)/examples/error===error3.go]
[4:$(pwd)/examples/error===error4.go]
...
...

Asynchronous Functions

In golang to perform asynchronous functions we use the keyword "go" it is responsible for putting the functions to be executed concurrently. A "go" statement starts the execution of a function call as an independent concurrent thread of control, or goroutine, within the same address space.

GoStmt = "go" Expression .

go Server()
go func(ch chan<- bool) { for { sleep(10); ch <- true }} (c)

Example:

package main

import (
	"fmt"
	"math/rand"
	"time"
)

var r = rand.Intn(500)

func pinger() {
	time.Sleep(time.Duration(r) * time.Microsecond)
	fmt.Println("pinger")
}

func ponger() {
	time.Sleep(time.Duration(r) * time.Microsecond)
	fmt.Println("ponger")
}

func printer() {
	time.Sleep(time.Duration(r) * time.Microsecond)
	fmt.Println("printer")
}

func main() {
	// making functions
	// calls asynchronously
	go pinger()
	go ponger()
	go printer()

	// Waiting to press any key to end
	var input string
	fmt.Scanln(&input)
}

Output one:

ponger
pinger
printer

Output two:

pinger
ponger
printer

Defer

A "defer" statement invokes a function whose execution is deferred to the moment the surrounding function returns, either because the surrounding function executed a return statement, reached the end of its function body, or because the corresponding goroutine is panicking.

DeferStmt = "defer" Expression .

The expression must be a function or method call; it cannot be parenthesized. Calls of built-in functions are restricted as for expression statements.

Each time a "defer" statement executes, the function value and parameters to the call are evaluated as usual and saved anew but the actual function is not invoked. Instead, deferred functions are invoked immediately before the surrounding function returns, in the reverse order they were deferred. If a deferred function value evaluates to nil, execution panics when the function is invoked, not when the "defer" statement is executed.

Examples:

defer unlock(l)
defer myFunc()
defer close(channel)
defer fmt.Print(x)
defer db.Close()
defer f.Close()
defer res.Body.Close()

Lab 03 Json with Golang

Json

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions.

Introduction

JSON (JavaScript Object Notation) is a simple data interchange format. Syntactically it resembles the objects and lists of JavaScript. It is most commonly used for communication between web back-ends and JavaScript programs running in the browser, but it is used in many other places, too. Its home page, json.org, provides a wonderfully clear and concise definition of the standard.

With the json package it's a snap to read and write JSON data from your Go programs.

Json Marshal Encode

Marshal returns the JSON encoding of v.

Marshal traverses the value v recursively. If an encountered value implements the Marshaler interface and is not a nil pointer, Marshal calls its MarshalJSON method to produce JSON. If no MarshalJSON method is present but the value implements encoding.TextMarshaler instead, Marshal calls its MarshalText method and encodes the result as a JSON string.

func Marshal(v interface{}) ([]byte, error)

Given the Go data structure, Message,

type ApiLogin struct {
   Name  string `json:"name"`
   Cpf   string `json:"cpf"`
}

and an instance of Message

m := ApiLogin{"Jefferson", "033.343.434-89"}

we can marshal a JSON-encoded version of m using json.Marshal:

b, err := json.Marshal(m)

If all is well, err will be nil and b will be a []byte containing this JSON data:

b == []byte(`{"Name":"Lambda Man","Cpf":"033.343.434-89"}`)

Check out the complete code:

import (
	"encoding/json"
	"fmt"
	"log"
)

type ApiLogin struct {
	Name string `json:"name"`
	Cpf  string `json:"cpf"`
}

func main() {

	a := ApiLogin{"Jefferson", "033.343.434-89"}
	fmt.Println(a)

	m, err := json.Marshal(a)
	if err != nil {
		log.Println(err)
	}
	// show bytes
	fmt.Println(m)

	// show string json
	fmt.Println(string(m))
}

Output:

{Jefferson 033.343.434-89}
[123 34 110 97 109 101 34 58 34 74 101 102 102 
101 114 115 111 110 34 44 34 99 112 102 34 58 
34 48 51 51 46 51 52 51 46 52 51 52 45 56 57 34 125]
{"name":"Jefferson","cpf":"033.343.434-89"}

Json MarshalIndent

MarshalIndent is like Marshal but applies Indent to format the output. Each JSON element in the output will begin on a new line beginning with prefix followed by one or more copies of indent according to the indentation nesting.

func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error)

Example:

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

type ApiLogin struct {
	Name string `json:"name"`
	Cpf  string `json:"cpf"`
}

func main() {

	a := ApiLogin{"Jefferson", "033.343.434-89"}
	// improving output for json format viewing
	json, err := json.MarshalIndent(a, "", "\t")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(json))
}

Output:

{
	"name": "Jefferson",
	"cpf": "033.343.434-89"
}

Option Omitempty

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

// Field appears in JSON as key "login".
Login string `json:"login"`

// Field appears in JSON as key "email" and
// the field is omitted from the object if its value is empty,
// as defined above.
Email string `json:"email,omitempty"`

// Field appears in JSON as key "nick" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Nick string `json:",omitempty"`

// Field is ignored by this package.
Level int `json:"-"`

// Field appears in JSON as key "-".
LastEmail string `json:"-,"`

Look at the example below:

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

type Login struct {
	// Field appears in JSON as key "login".
	Login string `json:"login"`

	// Field appears in JSON as key "email" and
	// the field is omitted from the object if its value is empty,
	// as defined above.
	Email string `json:"email,omitempty"`

	// Field appears in JSON as key "nick" (the default), but
	// the field is skipped if empty.
	// Note the leading comma.
	Nick string `json:",omitempty"`

	// Field is ignored by this package.
	Level int `json:"-"`

	// Field appears in JSON as key "-".
	LastEmail string `json:"-,"`
}

func main() {

	l := Login{Login: "Austin", Email: "[email protected]", Nick: 
	 "", Level: 1000, LastEmail: "[email protected]"}
	fmt.Println(l)

	m, err := json.Marshal(l)
	if err != nil {
		log.Println(err)
	}

	fmt.Println(string(m))
}

Output:

{Austin [email protected] Aust 1000 [email protected]}
{"login":"Austin","email":"[email protected]","-":"[email protected]"}

Only data structures that can be represented as valid JSON will be encoded:

- JSON objects only support strings as keys; to encode a Go map type it must be of 
  the form map[string]T (where T is any Go type supported by the json package).
- Channel, complex, and function types cannot be encoded.
- Cyclic data structures are not supported; they will cause Marshal to go into an infinite loop.
- Pointers will be encoded as the values they point to (or 'null' if the pointer is nil).

The json package only accesses the exported fields of struct types (those that begin with an uppercase letter). Therefore only the the exported fields of a struct will be present in the JSON output.

In this example we work with pointers to reference the struct within another struct, and another point is that we declare the struct within the struct itself. With this we have different ways to initialize and fill the fields of our structs. Let's see how it works? Check out the example below.

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

type Address struct {
	City         string `json:"city"`
	Neighborhood string `json:"neighborhood"`
	Zipcode      string `json:"zipcode"`
}

type ApiLogin struct {
	Name  string `json:"name"`
	Cpf   string `json:"cpf"`
	Login string `json:"login"`
	Email string `json:"email"`
	And1  *struct {
		City string
	}

	And2 *Address
}

func main() {

	apilogin1 := &ApiLogin{Name: "@jeffotoni", Cpf: "093.393.334-34",
		And1: &struct{ City string }{City: "BH"}, And2: &Address{City: "BH"}}
	//m, err := json.Marshal(apilogin1)
	m, err := json.MarshalIndent(apilogin1, "", "\t")

	if err != nil {
		log.Println(err)
	}
	//fmt.Println("apilogin1 initialized")
	//fmt.Println(apilogin1)

	//fmt.Println("\njson.Marshal returning bytes")
	//fmt.Println(m)

	fmt.Println("\njson.Marshal as string")
	fmt.Println(string(m))
}

json.Marshal as string
{
	"name": "@jeffotoni",
	"cpf": "093.393.334-34",
	"login": "",
	"email": "",
	"And1": {
		"City": "BH"
	},
	"And2": {
		"city": "BH",
		"neighborhood": "",
		"zipcode": ""
	}
}

In this other example we take a short excerpt from the json of an AWS SES service, it notifies via Json the Bounces of the sent emails, we are going to check the completion of our fields in the struct and transform them into json.

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

func main() {

	type BouncedRecipients []struct {
		EmailAddress string `json:"emailAddress"`
		Action       string `json:"action"`
		Status       string `json:"status"`
		//DiagnosticCode string `json:"diagnosticCode"`
	}

	type Bounce struct {
		BounceType string `json:"bounceType"`
		//BounceSubType string    `json:"bounceSubType"`
		//Timestamp     time.Time `json:"timestamp"`
		BR BouncedRecipients
	}

	type JsonMessage struct {
		NotificationType string `json:"notificationType"`
		B                Bounce
		From             []string `json:"from"`
	}

	l := &JsonMessage{NotificationType: "38733773737xxxx",
		B: Bounce{BounceType: "bounce type",
			BR: BouncedRecipients{
				{"[email protected]", "permanet", "error"}, {"[email protected]", "complaint", "success"}}},
		From: []string{"[email protected]", "[email protected]"}}

	m, err := json.Marshal(l)

	if err != nil {
		log.Println(err)
	}

	//fmt.Println("\033[1;40mlinkFetcher initialized\033[0m")
	//fmt.Println(l)

	//fmt.Println("\n\033[1;42mjson.Marshal returning bytes\033[0m")
	//fmt.Println(m)

	fmt.Println("\n\033[1;41mjson.Marshal as string\033[0m")
	fmt.Println(string(m))
}

Output:

json.Marshal as string
{
	"notificationType": "38733773737xxxx",
	"B": {
		"bounceType": "bounce type",
		"BR": [
			{
				"emailAddress": "[email protected]",
				"action": "permanet",
				"status": "error"
			},
			{
				"emailAddress": "[email protected]",
				"action": "complaint",
				"status": "success"
			}
		]
	},
	"from": [
		"[email protected]",
		"[email protected]"
	]
}

In the example below there is an entire field in lowercase, this field the json.Marshal function will not be able to do marshal, because the field initializes with the lowercase letter "myname", so it works the first letter has that is "Myname" Another legal point our struct is set to receive "N" values as an array of struct v[]struct" And inside the struct we have a string-type field "[]string". too cool is not? Let's see how we do the initialization and fill in the values, look at the code below.

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

type linkResult []struct {
	Body   string   `json:"body"`
	Urls   []string `json:"urls"`
	myname string   `json:"myname"`
}

func main() {
	
	// creating our struct
	// with some fields
	// interesting as [] string
	// and showing how to initialize them
	var ll = linkResult{
		{
			Body:   "The Go Programming Language",
			Urls:   []string{"https://golang.org/pkg/", "https://golang.org/cmd/"},
			myname: "lambda man",
		},
		{
			Body:   "Package",
			Urls:   []string{"https://golang.org/", "https://golang.org/cmd/", "https://golang.org/pkg/fmt/"},
			myname: "go_br in action",
		}}

	//m0, err := json.Marshal(ll)
	m0, err := json.MarshalIndent(ll, "", "\t")

	if err != nil {
		log.Println(err)
	}
	//fmt.Println("linkFetcher initialized")
	//fmt.Println(ll)

	//fmt.Println("\njson.Marshal returning bytes")
	//fmt.Println(m0)

	fmt.Println("\njson.Marshal as string")
	fmt.Println(string(m0))
}

Output:

linkFetcher initialized
[{The Go Programming Language [https://golang.org/pkg/ https://golang.org/cmd/] lambda man} {Package [https://golang.org/ https://golang.org/cmd/ https://golang.org/pkg/fmt/] go_br in action}]

json.Marshal returning bytes
[91 10 9 123 10 9 9 34 98 111 100 121 34 58 32 34 84 104 101 32 71 111 32 80 114 111 103 114 97 109 109 105 110 103 32 76 97 110 103 117 97 103 101 34 44 10 9 9 34 117 114 108 115 34 58 32 91 10 9 9 9 34 104 116 116 112 115 58 47 47 103 111 108 97 110 103 46 111 114 103 47 112 107 103 47 34 44 10 9 9 9 34 104 116 116 112 115 58 47 47 103 111 108 97 110 103 46 111 114 103 47 99 109 100 47 34 10 9 9 93 10 9 125 44 10 9 123 10 9 9 34 98 111 100 121 34 58 32 34 80 97 99 107 97 103 101 34 44 10 9 9 34 117 114 108 115 34 58 32 91 10 9 9 9 34 104 116 116 112 115 58 47 47 103 111 108 97 110 103 46 111 114 103 47 34 44 10 9 9 9 34 104 116 116 112 115 58 47 47 103 111 108 97 110 103 46 111 114 103 47 99 109 100 47 34 44 10 9 9 9 34 104 116 116 112 115 58 47 47 103 111 108 97 110 103 46 111 114 103 47 112 107 103 47 102 109 116 47 34 10 9 9 93 10 9 125 10 93]

json.Marshal as string
[
	{
		"body": "The Go Programming Language",
		"urls": [
			"https://golang.org/pkg/",
			"https://golang.org/cmd/"
		]
	},
	{
		"body": "Package",
		"urls": [
			"https://golang.org/",
			"https://golang.org/cmd/",
			"https://golang.org/pkg/fmt/"
		]
	}
]

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

type crud struct {
    Create   bool
    Retrieve bool
    Update   bool
    Delete   bool
}

type crudWithDetail struct {
    crud
    Detail bool
}

type acceleration struct {
    crudWithDetail
    Participant crudWithDetail
    Challenge   crudWithDetail
}

type acl struct {
    User         crud
    Acceleration acceleration
}

func main() {

    // on this boot we show exactly how it would be using non-embedded structs but all separated and making type of each one of them
    // all initialization is more friendly, although having nesting between structs is much better if we have built structs ie
    // structs being declared directly inside structs what they call embedded.

    // Example embedded strcut would
    // type my struct {
    //    M struct {
    //       Name string
    //   }
    // }

    j := acl{
        User: crud{Create: true, Retrieve: false, Update: false, Delete: true},
        Acceleration: acceleration{
            Participant:    crudWithDetail{Detail: true, crud: crud{Create: false, Retrieve: false, Update: true, Delete: false}},
            Challenge:      crudWithDetail{Detail: false, crud: crud{Create: true, Retrieve: true, Update: true, Delete: false}},
            crudWithDetail: crudWithDetail{Detail: false, crud: crud{Create: true, Retrieve: false, Update: true, Delete: false}}},
    }

    json, err := json.MarshalIndent(j, "", "\t")
    if err != nil {
        log.Println(err)
    }
    fmt.Println(string(json))
}

Output:

{
	"User": {
		"Create": true,
		"Retrieve": false,
		"Update": false,
		"Delete": true
	},
	"Acceleration": {
		"Create": true,
		"Retrieve": false,
		"Update": true,
		"Delete": false,
		"Detail": false,
		"Participant": {
			"Create": false,
			"Retrieve": false,
			"Update": true,
			"Delete": false,
			"Detail": true
		},
		"Challenge": {
			"Create": true,
			"Retrieve": true,
			"Update": true,
			"Delete": false,
			"Detail": false
		}
	}
}

Let's now present 3 ways to declare a map [string] interfaces {} inside a struct and generate a json of it all after initializing our struct.

Take a look at the example below:

package main

import (
	"encoding/json"
	"fmt"
)

// create type map
type body map[string]interface{}

// struct Message
type Message struct {
	Subject  string                 `json:"subject"`
	TimeSent int64                  `json:"sent"`
	Body     body                   `json:"body"`  // embedded => Body of the type map[string]interface{}
	Body2    map[string]interface{} `json:"body2"` // Body of the type map[string]interface{}
	Status   string                 `json:"status"`
}

func main() {

	// our interface map body
	// This map can be dynamic,
	// create types as needed, because the interface{}
	// is for this purpose, it creates dynamic types.
	b := body{
		"Data": body{
			"login":  "jeffotoni",
			"level":  1000,
			"create": true,
		}, "Address": body{
			"City": "Jersey City",
			"Zip":  "34.566-333",
			"Fone": body{
				"fone1": "87-77047345",
				"fone2": "83-93483838",
			},
		},
	}

	// create type
	type out map[string]interface{}

	// a way to initialize the struct
	// and the map contained within the struct
	s := &Message{
		Subject:  "Test Struct to Map",
		TimeSent: 345,
		Body:     b,
		Body2:    out{"Instance": "c5.xlarge", "vCpu": "4", "Ecu": "16", "Mem": "8"},
		Status:   "success",
	}
	fmt.Println(s)

	// Another way to implement is to generate
	// the map interface {} inside the initialization
	// of the struture.
	s2 := &Message{
		Subject:  "Test Struct to Map",
		TimeSent: 345,
		Body:     b,
		Body2:    map[string]interface{}{"Instance": "c5.xlarge", "vCpu": "4", "Ecu": "16", "Mem": "8"},
		Status:   "success",
	}
	fmt.Println(s2)

	// converting everything to Json
	m, err := json.Marshal(s2)
	if err != nil {
		fmt.Println(err)
	}

	// sending json to the screen
	fmt.Println("##### json")
	fmt.Println(string(m))
}

Output:

&{Test Struct to Map 345 map[Data:map[create:true login:jeffotoni level:1000] Address:map[City:Jersey City Zip:34.566-333 Fone:map[fone1:87-77047345 fone2:83-93483838]]] map[Instance:c5.xlarge vCpu:4 Ecu:16 Mem:8] success}
&{Test Struct to Map 345 map[Data:map[login:jeffotoni level:1000 create:true] Address:map[City:Jersey City Zip:34.566-333 Fone:map[fone1:87-77047345 fone2:83-93483838]]] map[Ecu:16 Mem:8 Instance:c5.xlarge vCpu:4] success}
##### json
{
	"subject": "Test Struct to Map",
	"sent": 345,
	"body": {
		"Address": {
			"City": "Jersey City",
			"Fone": {
				"fone1": "87-77047345",
				"fone2": "83-93483838"
			},
			"Zip": "34.566-333"
		},
		"Data": {
			"create": true,
			"level": 1000,
			"login": "jeffotoni"
		}
	},
	"body2": {
		"Ecu": "16",
		"Instance": "c5.xlarge",
		"Mem": "8",
		"vCpu": "4"
	},
	"status": "success"
}

Initialized Collections Of Data

Some ways to anonymously declare and initialize types and collections of types in marshal to transform into Json.

Take a look at the example below:

func main() {

    // let's try something simple
    // to understand what's
    // really going on
    m, _ := json.Marshal(false)
    fmt.Println(m)         // show as bytes
    fmt.Println(string(m)) // show as string

    m2, _ := json.Marshal(-1)
    fmt.Println(string(m2))

    m3, _ := json.Marshal(0)
    fmt.Println(string(m3))

    m4, _ := json.Marshal(102.3)
    fmt.Println(string(m4))

    m5, _ := json.Marshal("DevOpsBH")
    fmt.Println(string(m5))

    // For this to occur we need to pass a collection
    // with fields, something like a struct, a slice,
    // maps, interfaces {} Let's see below an example
    m6, _ := json.Marshal([...]string{"Elixir", "Scala"})
    fmt.Println(string(m6))

    m7, _ := json.Marshal(map[string]string{"twitter": "@jeffotoni"})
    fmt.Println(string(m7))

    m8, _ := json.Marshal(map[string]interface{}{"instagram": "jeffotoni", "langs": struct{ G string }{G: "gophers"}})
    fmt.Println(string(m8))

    m9, _ := json.Marshal(map[string]struct{ L string }{"jeff": {L: "@Ricardo Maraschini"}})
    fmt.Println(string(m9))
}

Output:

Json NewEncoder

NewEncoder returns a new encoder that writes to w.

func NewEncoder(w io.Writer) *Encoder

With this function we can implement our own custom Marshal function

Check out the example below:

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
)

const (
    empty = ""
    tab   = "\t"
)

func main() {

    //json, err := MyJson(map[string]string{"twitter": "@jeffotoni"})
    json, err := MyJson(map[string]interface{}{"instagram": "jeffotoni", "langs": struct{ G string }{G: "gophers"}})

    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(json)
}

func MyJson(data interface{}) (string, error) {
    buffer := new(bytes.Buffer)
    encoder := json.NewEncoder(buffer)
    encoder.SetIndent(empty, tab)
    // encode...
    err := encoder.Encode(data)
    if err != nil {
        return empty, err
    }
    return buffer.String(), nil
}

Output:

{
	"instagram": "jeffotoni",
	"langs": {
		"G": "gophers"
	}
}

The above example shows the use of the json.NewEncoder () function. Encode () to transform the data collection into JSON, very cool, right?

Json Unmarshal Decode

Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v. If v is nil or not a pointer, Unmarshal returns an "InvalidUnmarshalError".

Unmarshal uses the inverse of the encodings that Marshal uses, allocating maps, slices, and pointers as necessary, with the following additional rules:

To unmarshal JSON into a pointer, Unmarshal first handles the case of the JSON being the JSON literal null. In that case, Unmarshal sets the pointer to nil. Otherwise, Unmarshal unmarshals the JSON into the value pointed at by the pointer. If the pointer is nil, Unmarshal allocates a new value for it to point to.

To unmarshal JSON into a value implementing the Unmarshaler interface, Unmarshal calls that value's UnmarshalJSON method, including when the input is a JSON null. Otherwise, if the value implements encoding.TextUnmarshaler and the input is a JSON quoted string, Unmarshal calls that value's UnmarshalText method with the unquoted form of the string.

To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal (either the struct field name or its tag), preferring an exact match but also accepting a case-insensitive match. By default, object keys which don't have a corresponding struct field are ignored (see Decoder.DisallowUnknownFields for an alternative).

To unmarshal a JSON array into a slice, Unmarshal resets the slice length to zero and then appends each element to the slice. As a special case, to unmarshal an empty JSON array into a slice, Unmarshal replaces the slice with a new empty slice.

Consider this JSON data, stored in the variable b:

b := []byte(`{"Name":"DevOps","Age":10,"Parents":["Google","Aws"]}`)

Without knowing this data's structure, we can decode it into an interface{} value with Unmarshal:

var f interface{}
err := json.Unmarshal(b, &f)

To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:

bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null

All of the content below has been described and created examples using unmarshal with interfaces {} which is one of the most complex ways of handling a value that we do not know the type they can be born dynamically, and for this we need to understand a little more how interfaces { } and how to do type assertions in Go.

Soon after we will return in unmarshal using structs.

Generic JSON with Interface{} and Assertion

The interface{} (empty interface) type describes an interface with zero methods. Every Go type implements at least zero methods and therefore satisfies the empty interface.

The empty interface serves as a general container type:

var i interface{}
i = "DevOps BH"
i = 2019
i = 9.5

A type assertion accesses the underlying concrete type:

r := i.(float64)
fmt.Println("the circle's area", math.Pi*r*r)

Or, if the underlying type is unknown, a type switch determines the type:

switch v := i.(type) {
case int:
    fmt.Println("twice i is", v*2)
case float64:
    fmt.Println("the reciprocal of i is", 1/v)
case string:
    h := len(v) / 2
    fmt.Println("i swapped by halves is", v[h:]+v[:h])
default:
    // i isn't one of the types above
}

The code below was perfect for us to understand, and to know how we make insertions using Golang.

Example:

package main

import "fmt"

func main() {
	var i interface{} = "DevOpsBh"

	s := i.(string)
	fmt.Println(s)

	s, ok := i.(string)
	fmt.Println(s, ok)

	f, ok := i.(float64)
	fmt.Println(f, ok)

	// f = i.(float64) // panic
	// fmt.Println(f)
}

Output:

DevOpsBh
DevOpsBh true
0 false

Type Switches

A type switch is a construct that permits several type assertions in series. A type switch is like a regular switch statement, but the cases in a type switch specify types (not values), and those values are compared against the type of the value held by the given interface value.

switch v := i.(type) {
case T:
    // here v has type T
case S:
    // here v has type S
default:
    // no match; here v has the same type as i
}

The declaration in a type switch has the same syntax as a type assertion i.(T), but the specific type T is replaced with the keyword type.

Check out the code below:

package main

import "fmt"

func doInterface(i interface{}) {
	switch v := i.(type) {
	case int:
		fmt.Printf("Twice %v is %v\n", v, v*2)
	case string:
		fmt.Printf("%q is %v bytes long\n", v, len(v))
	default:
		fmt.Printf("I don't know about type %T!\n", v)
	}
}

func main() {
	doInterface(33)
	doInterface("DevOpsBH")
	doInterface("Lambda")
	doInterface(true)
}

Output:

Twice 33 is 66
"DevOpsBH" is 8 bytes long
"Lambda" is 6 bytes long
I don't know about type bool!

The code below will initialize the interface causing it to receive several different types.

Check out all the code below:

package main

import "fmt"

func main() {
	var i interface{}
	fmt.Println(i)

	i = "DevOps BH"
	fmt.Println(i)

	i = 2019
	fmt.Println(i)

	i = 9.5
	fmt.Println(i)

	i = [...]string{"Pike", "Jeffotoni", "Thompson", "Griesemer"}
	fmt.Println(i)

	i = map[string]interface{}{"Lang": []string{"Go", "Rust", "Scala", "Elixir"}}
	fmt.Println(i)

	//struct{ City string }{City: "BH"}
	i = map[struct{ L string }]interface{}{{L: "Lang"}: []string{"Go", "Rust", "Scala", "Elixir"}}
	fmt.Println(i)
}

Output:

<nil>
DevOps BH
2019
9.5
[Pike Jeffotoni Thompson Griesemer]
map[Lang:[Go Rust Scala Elixir]]
map[{Lang}:[Go Rust Scala Elixir]]

It is noticed that the interface created accepted all the types that was passed to it, was created dynamically.

Syntax of Type Assertion is Defined as:

PrimaryExpression.(Type)

At this point the Go value in f would be a map whose keys are strings and whose values are themselves stored as empty interface values:

f = map[string]interface{}{
    "Name": "DevOps",
    "Age":  10,
    "Parents": []interface{}{
        "Google",
        "Aws",
    },
}

To access this data we can use a type assertion to access f's underlying map[string]interface{}:

m := f.(map[string]interface{})

We can then iterate through the map with a range statement and use a type switch to access its values as their concrete types:

for k, v := range m {
    switch vv := v.(type) {
    case string:
        fmt.Println(k, "is string", vv)
    case float64:
        fmt.Println(k, "is float64", vv)
    case []interface{}:
        fmt.Println(k, "is an array:")
        for i, u := range vv {
            fmt.Println(i, u)
        }
    default:
        fmt.Println(k, "is of a type I don't know how to handle")
    }
}

In this way you can work with unknown JSON data while still enjoying the benefits of type safety.

See full code below:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {

	b := []byte(`{"Name":"DevOps","Age":10,"Company":["Google","Aws"]}`)

	var f interface{}
	if err := json.Unmarshal(b, &f); err != nil {
		fmt.Println(err)
	}

	fmt.Println(f)

	m := f.(map[string]interface{})
	for k, v := range m {
		switch vv := v.(type) {
		case string:
			fmt.Println(k, "is string", vv)
		case float64:
			fmt.Println(k, "is float64", vv)
		case []interface{}:
			fmt.Println(k, "is an array:")
			for i, u := range vv {
				fmt.Println(i, u)
			}
		default:
			fmt.Println(k, "is of a type I don't know how to handle")
		}
	}
	//fmt.Println()
}

map[Name:DevOps Age:10 Company:[Google Aws]]
Name is string DevOps
Age is float64 10
Company is an array:
0 Google
1 Aws

When we work with empty interfaces, we have to do asserts, so we can use them and treat them correctly, because in Golang everything is static and typed, if we do not assert we will not be able to capture the values of types that are in the interface that can be any type.

Check the code below:

	case string:
	// ..
	case float64:
	// ..
	case [] interface {}:
	// ..
	case int:
	// ..
	case bool:
	// ..
	case map[string]interface{}:

Dynamic Type

Besides static type that all variables have (it’s a type from variable’s declaration), variables of interface type also have a dynamic type. It’s a type of value currently set in interface type variable. Over the course of program execution variable of interface type has the same static type but its dynamic type can change as different values implementing desired interface will be assigned:

Check out the example:

package main

import "fmt"

type I interface {
	comeon()
}

type A struct{}

func (a A) comeon() {}

type B struct{}

func (b B) comeon() {}

func main() {
	var i I
	i = A{} // dynamic type of i is A
	fmt.Printf("%T\n", i.(A))
	i = B{} // dynamic type of i is B
	fmt.Printf("%T\n", i.(B))
}

main.A
main.B

What is Reflection

Reflection is the ability of a program to inspect its variables and values at run time and find their type. You might not understand what this means but that's alright. You will get a clear understanding of reflection by the end of this section, so stay with me.

Reflection is a very powerful and advanced concept in Go and it should be used with care. It is very difficult to write clear and maintainable code using reflection. It should be avoided wherever possible and should be used only when absolutely necessary.

This is very powerful, we managed to sweep our entire struct using asserts and reflect, we got the name of the struct, name of the fields, their values, their tags, this feature is used to do Parse in Files, like Yaml, Toml, Json etc ......

Check the code below:

import (
	"encoding/json"
	"fmt"
)

func main() {

	b := []byte(`{"Name":"Pike Hob","Age":56,"Parents":["Denis","James"]}`)
	var f interface{}
	_ = json.Unmarshal(b, &f)

	m := f.(map[string]interface{})

	// a way to initialize a map interface
	////////////////////////////////////////

	// f = map[string]interface{}{
	// 	"Name": "Wednesday",
	// 	"Age":  6,
	// 	"Parents": []interface{}{
	// 		"Torvalds",
	// 		"Mark Zuckerberg",
	// 	},
	// }
	/////////////////////////////////////

	for k, v := range m {
		switch vv := v.(type) {
		case string:
			fmt.Println(k, "|is string|", vv)
		case float64:
			fmt.Println(k, "|is float64|", vv)
		case []interface{}:
			fmt.Println(k, "|is an array|")
			for i, u := range vv {
				fmt.Println(i, u)
			}
		default:
			fmt.Println(k, "is of a type I don't know how to handle")
		}
	}
}

Output:

Age |is float64| 56
Parents |is an array|
0 Denis
1 James
Name |is string| Pike Hob

Let's take another example when using Unmarshal in Interfaces{} Example:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	// map more interface
	input := map[string]interface{}{
		"Key1": []string{"some", "key"},
		"key3": nil,
		"val":  2,
		"val2": "str",
		"val3": 4,
	}

	fmt.Println(input)
	for key, value := range input {
		slice, ok := value.([]string)
		if ok {
			input["Count"+key] = len(slice)
		}
	}
	fmt.Println(input)

	// Encode to Json
	m, err := json.Marshal(input)
	if err != nil {
		fmt.Println(err)
	}

	// convert byte to string
	fmt.Println(string(m))

	// interface empty
	var f interface{}

	// Decode json
	if err := json.Unmarshal(m, &f); err != nil {
		fmt.Println(err)
	}

	// show screen
	fmt.Println(f)

	fmt.Println("### loop interface")
	m2 := f.(map[string]interface{})
	loopI(m2)
}

// loop in interface
func loopI(m2 map[string]interface{}) {
	for k, v := range m2 {

		// assert interface
		switch vv := v.(type) {
		case string:
			fmt.Println(k, "is string", vv)
		case float64:
			fmt.Println(k, "is float64", vv)
		case []interface{}:
			fmt.Println(k, "is an array:")
			for i, u := range vv {
				fmt.Println(i, u)
			}
		default:
			fmt.Println(k, "is of a type I don't know how to handle")
		}
	}
}

Output:

map[key3:<nil> Key1:[some key] val:2 val2:str val3:4]
map[key3:<nil> Key1:[some key] val:2 val2:str val3:4 CountKey1:2]
{"CountKey1":2,"Key1":["some","key"],"key3":null,"val":2,"val2":"str","val3":4}
map[CountKey1:2 Key1:[some key] key3:<nil> val:2 val2:str val3:4]
### loop interface
Key1 is an array:
0 some
1 key
key3 is of a type I don't know how to handle
val is float64 2
val2 is string str
val3 is float64 4
CountKey1 is float64 2

Making Reflect with Struct

The example below clearly shows how we would do a reflect in a struct in a simple way. We were able to sweep the struct at element level, we get the Struct Name, the Field Name the field types, the tags in the fields and their values when they are filled. Phew, a lot of cool stuff there.

Let's see the code below:

package main

import (
	"fmt"
	"reflect"
)

type User struct {
	FirstName string `tag_name:"firstname"`
	LastName  string `tag_name:"lastname"`
	Age       int    `tag_name:"age"`
}

func (f *User) reflect() {

	// slice the element of struct
	val := reflect.ValueOf(f).Elem()

	// loop in elemnts of struct
	for i := 0; i < val.NumField(); i++ {

		// value of interface
		valueField := val.Field(i)

		// object of struct
		typeField := val.Type().Field(i)

		// tag of field
		tag := typeField.Tag
		fmt.Printf("Field Name: %s,\t Field Value: %v,\t Tag Value: %s\n", typeField.Name, valueField.Interface(), tag.Get("tag_name"))
	}
}

func main() {

	f := &User{
		FirstName: "Jefferson",
		LastName:  "Otoni",
		Age:       350,
	}

	f.reflect()
}

Output:

Field Name: FirstName,	 Field Value: Jefferson,	 Tag Value: firstname
Field Name: LastName,	 Field Value: Otoni,	 Tag Value: lastname
Field Name: Age,	 Field Value: 350,	 Tag Value: age

Let's see how we would do an array dump using reflect.

Check out the complete code below:

package main

import (
	"fmt"
	"reflect"
)

func dump_interface_array(args interface{}) {

	// getting the interface
	val := reflect.ValueOf(args)

	// test the type
	if val.Kind() == reflect.Array {
		fmt.Println("len = ", val.Len())
		for i := 0; i < val.Len(); i++ {
			e := val.Index(i)
			switch e.Kind() {
			case reflect.Int:
				fmt.Printf("%v, ", e.Int())
			case reflect.Float32:
				fallthrough
			case reflect.Float64:
				fmt.Printf("%v, ", e.Float())
			case reflect.String:
				fmt.Printf("%v, ", e.String())
			default:
				panic(fmt.Sprintf("invalid Kind: %v", e.Kind()))
			}
		}
		fmt.Println()
	}
}

func main() {

	// array int
	int_ary := [4]int{1, 2, 3, 4}

	// array float
	float32_ary := [4]float32{1.1, 2.2, 3.3, 4.4}

	// array float
	float64_ary := [4]float64{1.1, 2.2, 3.3, 4.4}

	// array string
	string_ary := [...]string{"Scala", "Elixir", "Lisp", "Clojure"}

	dump_interface_array(int_ary)
	dump_interface_array(float32_ary)
	dump_interface_array(float64_ary)
	dump_interface_array(string_ary)
}

Output:

len =  4
1, 2, 3, 4,
len =  4
1.100000023841858, 2.200000047683716, 3.299999952316284, 4.400000095367432, 
len =  4
1.1, 2.2, 3.3, 4.4, 
len =  4
Scala, Elixir, Lisp, Clojure,

Another way to sweep dynamic slices, cool, let's check.

Look at the complete code:

package main

import "fmt"
import "reflect"

func main() {

	// slice dinamic
	slice := []string{"C", "C++", "Fortram", "Cobol"}
	dump_slice(slice)

	// slice int
	dataint := []int{1, 2, 3}
	dump_slice(dataint)
}

// dump interfaces
func dump_slice(t interface{}) {

	// type kind only slice
	switch reflect.TypeOf(t).Kind() {

	// slice
	case reflect.Slice:

		// return interface
		s := reflect.ValueOf(t)

		// loop in type
		for i := 0; i < s.Len(); i++ {
			fmt.Println(s.Index(i))
		}
	}
}

Output:

C
C++
Fortram
Cobol
1
2
3

Below we have an example doing some types of reflect, simple but for us to have a good idea of reflect power.

Let's see the complete code below:

package main

import (
    "fmt"
    "reflect"
)

func main() {

    // map interface
    Schema := map[string]interface{}{}

    // let's fill in the fields
    Schema["int"] = 10
    Schema["string"] = "this is a string and map with interface"
    Schema["bool"] = false
    Schema["sliceString"] = [...]string{"C", "C++", "Lisp"}

    // interface
    val := reflect.ValueOf(Schema)

    // all data
    fmt.Println("Schema", val)

    // type
    fmt.Println("Type = ", val.Kind())

    // kind type == reflect map
    if val.Kind() == reflect.Map {
        for _, key := range val.MapKeys() {
            v := val.MapIndex(key)

            // kind type, interface type
            switch value := v.Interface().(type) {

            // testing types
            case int:
                fmt.Println(key, value)
            case string:
                fmt.Println(key, value)
            case bool:
                fmt.Println(key, value)

            // case map[string]string:

            // case []interface{}:

            // case map[string]interface{}:

            default:
                val2 := reflect.ValueOf(Schema["sliceString"])
                // type array
                if val2.Kind() == reflect.Array {
                    //fmt.Println("len = ", val2.Len())

                    // loop array
                    for i := 0; i < val2.Len(); i++ {
                        // index key
                        e := val2.Index(i)

                        // kind type
                        switch e.Kind() {

                        // case types
                        case reflect.String:
                            fmt.Printf("%v, ", e.String())

                        //default
                        default:
                            panic(fmt.Sprintf("invalid Kind: %v", e.Kind()))
                        }
                    }
                    fmt.Println()
                }
            }
        }
    }
}

Output:

Schema map[int:10 string:this is a string and map with interface bool:false sliceString:[C C++ Lisp]]
Type =  map
int 10
string this is a string and map with interface
bool false
C, C++, Lisp,

Now let's complicate it a bit, let's do a reflect read of a struct and transform it into a map. Inside the struct we put a map of interfaces just to see how it would look.

Check the code below:

package main

import (
	"fmt"
	"net/url"
	"reflect"
	"strconv"
)

type address map[string]interface{}

type User struct {
	Login   string  `json:"login"`
	Email   string  `json:"email"`
	Status  string  `json:"status"`
	Address address `json:"address"`
}

func main() {

	a := address{
		"Address": address{
			"City": "Belo Horizonte",
			"Zip":  "34.566-333",
			"Fone": address{
				"fone1": "87-77047345",
				"fone2": "83-93483838",
			},
		},
	}

	user := User{
		Login:   "jeffotoni",
		Email:   "[email protected]",
		Status:  "alive",
		Address: a,
	}

	urlValues := structToMap(&user)
	fmt.Println(urlValues)
}

// convert struct to map
func structToMap(i interface{}) (values url.Values) {
	values = url.Values{}
	iVal := reflect.ValueOf(i).Elem()
	typ := iVal.Type()
	for i := 0; i < iVal.NumField(); i++ {
		f := iVal.Field(i)
		// You ca use tags here...
		// tag := typ.Field(i).Tag.Get("tagname")
		// Convert each type into a string for the url.Values string map
		var v string
		switch f.Interface().(type) {
		case int, int8, int16, int32, int64:
			v = strconv.FormatInt(f.Int(), 10)
		case uint, uint8, uint16, uint32, uint64:
			v = strconv.FormatUint(f.Uint(), 10)
		case float32:
			v = strconv.FormatFloat(f.Float(), 'f', 4, 32)
		case float64:
			v = strconv.FormatFloat(f.Float(), 'f', 4, 64)
		case []byte:
			v = string(f.Bytes())
		case string:
			v = f.String()

		default:

			switch f.Kind() {
			// map
			case reflect.Map:
				for _, key := range f.MapKeys() {
					strct := f.MapIndex(key)
					//fmt.Println(key.Interface(), strct.Interface())
					v = fmt.Sprintf("%v %v", key.Interface(), strct.Interface())

				}
			}
		}

		values.Set(typ.Field(i).Name, v)
	}
	return
}

Output:

map[Login:[jeffotoni] Email:[[email protected]] Status:[alive] 
Address:[Address map[City:Belo Horizonte Zip:34.566-333 
Fone:map[fone1:87-77047345 fone2:83-93483838]]]]

Json Unmarshal Structs

We saw the most complex part of using Unmarshal in dynamic composites using interfaces {} and had to make assertions to capture the values. Now let's use Unmarshal in structures, in this scenario the structure is something static, it has to be always predefined.

When we develop our API, we will realize that we will have to do Unmarshal and Marshal all the time of receiving and sending information using json.

So if we get angry at Unmarshal, Marshal, Interfaces {} and Structs, our APIs will be a little tricky to develop.

Let's take a look at the code below that represents exactly the way we use Unmarshal with Structs.

package main

import (
	"encoding/json"
	"fmt"
)

type jsoninput []struct {
	Name string `json:"name"`
}

func main() {

	// json in memory
	// this is an array of values
	resp := `[{"name":"Andre"},{"name":"Pike"}]`

	// initialization struct
	data := &jsoninput{}

	// Unmarshal in bytes
	_ = json.Unmarshal([]byte(resp), data)

	// loop to see the values in the fields
	// loop in struct
	for _, value := range *data {
		fmt.Println(value.Name)
	}
}

Output:

Andre
Pike

Example:

import (
	"encoding/json"
	"fmt"
)

func main() {

	// byte with json
	var jsonBlob = []byte(`[
	{"Name": "Golang", "Level": "10"},
	{"Name": "Rust",    "Level": "7"}
]`)

	// struct that is our model
	type Lang struct {
		Name  string
		Level string
	}

	var langs []Lang

	// convert Json to Struct
	err := json.Unmarshal(jsonBlob, &langs)
	if err != nil {
		fmt.Println("error:", err)
	}

	fmt.Printf("\n%+v", langs)
	fmt.Printf("\n%+v", langs[0].Name)
	fmt.Printf("\n%+v", langs[0].Level)
	fmt.Printf("\n%+v", langs[1].Name)
	fmt.Printf("\n%+v", langs[1].Level)
}

Output:

[{Name:Golang Level:10} {Name:Rust Level:7}]
Golang
10
Rust
7

In other words, we get a string of bytes that is a Json and fill in the values of the struct with those elements. Using the json.Unmarshal function, Fantastic it. In other words, Json has to be compatible with the struct that is our model.

MarshalJSON and UnmarshalJSON

There is the possibility of customizing the native method of Marshal and Unmarshal, that is to override with other functionalities without changing their behavior as a whole, but to increase in the already existing method. Very cool this feature, I do not need to rewrite the whole method in yes add a behavior.

First we create our struct with the fields we want to add the new rule as the example below:

type Email struct {
Email string
}

Soon after the initialization of my struct where it feeds the information will receive this struct as parameter as the example below:

a: = struct {
Login string
Email
Time Time
}

Ready now we simply implement our method always with the name MarshalJSON referencing this struct that we created "Email".

func (and Email) MarshalJSON () ([] byte, error) {
return [] byte (fmt.Sprintf (`"% s "`, checkEmail (e.Email))), nil
}

The checkEmail function is a regular expression that we created to validate the email. Ready when running the Marshal function it will icorporate this method that you implemented. See the example below with the complete code.

package main

import (
	"encoding/json"
	"fmt"
	"regexp"
	"time"
)

type Email struct {
	Email string
}

type Time struct {
	time.Time
}

// layout
const layout = "2006-01-02"

var regxmail = regexp.MustCompile("^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$")

func (t Time) MarshalJSON() ([]byte, error) {
	return []byte(fmt.Sprintf(`"%s"`, t.Time.Format(layout))), nil
}

func (e Email) MarshalJSON() ([]byte, error) {
	return []byte(fmt.Sprintf(`"%s"`, checkEmail(e.Email))), nil
}

func checkEmail(email string) string {
	if regxmail.MatchString(email) {
		return email
	} else {
		return "-invalid-"
	}
}

func main() {
	a := struct {
		Login string
		Email Email
		Time  Time
	}{
		Login: "devops",
		Email: Email{"jeffotoni-go.com"},
		Time:  Time{time.Now()},
	}

	fmt.Println(a)
	json, err := json.MarshalIndent(a, "", "\t")
	if err != nil {
		fmt.Println(err)
	}

	fmt.Println(string(json))
}

Output:

{devops {jeffotoni-go.com} 2019-01-30 22:41:00.506602023 -0200 -02 m=+0.000508436}
{
	"Login": "devops",
	"Email": "-invalid-",
	"Time": "2019-01-30"
}

The Unmarshal version is a little more complex, since we will need to scan the received structures. The email was added in a string map, to scan the field and add the functionality to validate the email.

Check out the complete code below.

package main

import (
	"encoding/json"
	"fmt"
	"regexp"
	"strings"
	"time"
)

type Email struct {
	Email string
}

type Time struct {
	time.Time
}

// layout
const layout = "2006-01-02"

// regex to email
var regxmail = regexp.MustCompile("^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$")

func (t *Time) UnmarshalJSON(b []byte) (err error) {
	if b[0] == '"' && b[len(b)-1] == '"' {
		b = b[1 : len(b)-1]
	}
	if string(b) == `null` {
		*t = Time{}
		return
	}
	t.Time, err = time.Parse(layout, string(b))
	return
}

// Unmarshal Json
func (t *Email) UnmarshalJSON(b []byte) (err error) {
	if b[0] == '"' && b[len(b)-1] == '"' {
		b = b[1 : len(b)-1]
	}
	if string(b) == `null` {
		return
	}
	var stuff map[string]string
	err = json.Unmarshal(b, &stuff)
	if err != nil {
		return err
	}
	for key, value := range stuff {
		if strings.ToLower(key) == "email" {
			t.Email = checkEmail(value)
		}
	}
	return nil
}

func checkEmail(email string) string {
	if regxmail.MatchString(email) {
		return email
	} else {
		return "-invalid-"
	}
}

func main() {
	a := struct {
		Login string
		Email Email
		Time  Time
	}{}

	b := []byte(`{"Login":"devops","Email":{"Email":"devops-go.com"},"Time":"2019-01-30"}`)
	err := json.Unmarshal(b, &a)
	if err != nil {
		fmt.Println(err)
	}
	fmt.Println(a)
}

Output:

{devops {-invalid-} 2019-01-30 00:00:00 +0000 UTC}

Parse Json

Reading and Parsing a JSON File

Let's do parse in Json file, there are several libs to do this, let's create our own, simple to understand the whole process and of course get mad at Parse.

The file below is saved in the folder parsejson and we will create our parse within this directory.

{
  "users": [
    {
      "name" : "Devopsbh",
      "type" : "Reader",
      "age" : 29,
      "social" : {
        "facebook" : "https://facebook.com/devopsbh",
        "twitter" : "https://twitter.com/devopsbh",
        "instagram" : "https://instagram.com/devopsbh"
      }
    },
    {
      "name" : "Jefferson",
      "type" : "Author",
      "age" : 160,
      "social" : {
        "facebook" : "https://facebook.com/jeffotoni",
        "twitter" : "https://twitter.com/jeffotoni"
        "instagram" : "https://instagram.com/jeffotoni"
      }
    }
  ]
}

We'll be using the packages in order to open up our users.json file from our filesystem. Once we have opened the file, we'll defer the closing of the file to the end of the function so that we can work with the data inside of it.

Check the code below:

package main

import (
	"fmt"
	"os"
)

func main() {
	//fmt.Println(os.Getwd())
	// Open our jsonFile
	jsonFile, err := os.Open("./users.json")

	// if we os.Open returns an error then handle it
	if err != nil {
		fmt.Println(err)
	}
	fmt.Println("Successfully Opened users.json")
	// defer the closing of our jsonFile so that we can parse it later on
	defer jsonFile.Close()
}

Parsing with Structs

We have a few options when it comes to parsing the JSON that is contained within our users.json file. We could either unmarshal the JSON using a set of predefined structs, or we could unmarshal the JSON using a map[string]interface{} to parse our JSON into strings mapped against arbitrary data types.

If you know the structure that you are expecting then I would recommend going down the verbose route and defining your structs like so:

package main

func main() {
	// Users struct which contains
	// an array of users
	type Users struct {
		Users []User `json:"users"`
	}

	// Social struct which contains a
	// list of links
	type Social struct {
		Facebook string `json:"facebook"`
		Twitter  string `json:"twitter"`
		Instagram  string `json:"instagram"`
	}

	// User struct which contains a name
	// a type and a list of social links
	type User struct {
		Name   string `json:"name"`
		Type   string `json:"type"`
		Age    int    `json:"Age"`
		Social Social `json:"social"`
	}
}

Once we have these in place, we can use them to unmarshal our JSON. Unmarshalling our JSON - Once we’ve used the os.Open function to read our file into memory, we then have to convert it toa byte array using ioutil.ReadAll. Once it’s in a byte array we can pass it to our json.Unmarshal() method.

Remember if the json file is a wrong comma, it will not convert the file to json format that has it right. Now you can check the code below.

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"os"
	"strconv"
)

// Users struct which contains
// an array of users
type Users struct {
	Users []User `json:"users"`
}

// Social struct which contains a
// list of links
type Social struct {
	Facebook  string `json:"facebook"`
	Twitter   string `json:"twitter"`
	Instagram string `json:"instagram"`
}

// User struct which contains a name
// a type and a list of social links
type User struct {
	Name   string `json:"name"`
	Type   string `json:"type"`
	Age    int    `json:"Age"`
	Social Social `json:"social"`
}

func main() {

	//fmt.Println(os.Getwd())
	// Open our jsonFile
	jsonFile, err := os.Open("./users.json")
	if err != nil {
		fmt.Println(err)
		return
	}
	defer jsonFile.Close()

	// read our opened xmlFile as a byte array.
	byteValue, _ := ioutil.ReadAll(jsonFile)

	// show json
	fmt.Println(string(byteValue))

	// we initialize our Users array
	var users Users

	// we unmarshal our byteArray which contains our
	// jsonFile's content into 'users' which we defined above
	json.Unmarshal(byteValue, &users)

	fmt.Println(users)

	// we iterate through every user within our users array and
	// print out the user Type, their name, and their facebook url
	// as just an example
	for i := 0; i < len(users.Users); i++ {
		fmt.Println("User Type: " + users.Users[i].Type)
		fmt.Println("User Age: " + strconv.Itoa(users.Users[i].Age))
		fmt.Println("User Name: " + users.Users[i].Name)
		fmt.Println("Facebook Url: " + users.Users[i].Social.Facebook)
		fmt.Println("Twitter Url: " + users.Users[i].Social.Twitter)
		fmt.Println("Instagram Url: " + users.Users[i].Social.Instagram)
	}
}

Output:

{[{Devopsbh Reader 29 {https://facebook.com/devopsbh https://twitter.com/devopsbh https://instagram.com/devopsbh}} {Jefferson Author 160 {https://facebook.com/jeffotoni https://twitter.com/jeffotoni https://instagram.com/jeffotoni}}]}
User Type: Reader
User Age: 29
User Name: Devopsbh
Facebook Url: https://facebook.com/devopsbh
Twitter Url: https://twitter.com/devopsbh
Instagram Url: https://instagram.com/devopsbh
User Type: Author
User Age: 160
User Name: Jefferson
Facebook Url: https://facebook.com/jeffotoni
Twitter Url: https://twitter.com/jeffotoni
Instagram Url: https://instagram.com/jeffotoni

Parsing with Map and Interface

Sometimes, going through the process of creating structs for everything can be somewhat time consuming and overly verbose for the problems you are trying to solve. In this instance, we can use standard interfaces{} in order to read in any JSON data:

Format Json Example:

{
  "name" : "Devopsbh",
  "city" : "Belo Horizonte",
  "age" : 29
}

How can we do Unmarshal in this file and access the fields?

When we read json and do the parse, there are several boring points we have to deal with, and an excellent strategy is to divide to conquer. The above example is an arrayless json with no complexity, a json in its simplest possible format.

Our Unmarshal received var result map[string]interface{} to be able to play all the data in an interface map so we have something dynamic without having to define the fields the interface makes the magic of accepting any type. In this example we have no problem in directly accessing the map fields, in json format we have the keys and values and ready we can direct access.

Check the code below:

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
)

func main() {

	// Open our jsonFile
	byteJson, err := ioutil.ReadFile("./user-only-0.json")

	// if we os.Open returns an error then handle it
	if err != nil {
		fmt.Println(err)
		return
	}

	// show screen
	fmt.Println("Successfully Opened users.json")

	// defining the interface map that will receive
	// the file and the format is dynamic
	var result = make(map[string]interface{})

	// let's now run our Unmarshal and convert to objects
	json.Unmarshal(byteJson, &result)

	// show all
	fmt.Println(result)

	// how is a map I can do exactly
	// the syntax below
	val, ok := result["name"].(string)
	fmt.Println(val, ok)

	// type string
	fmt.Println(result["name"].(string))

	// type string
	fmt.Println(result["city"].(string))

	// type float64
	fmt.Println(result["age"].(float64))
}

Output:

Successfully Opened users.json
map[age:3 name:Devopsbh city:Belo Horizonte]
Devopsbh true
Devopsbh
Belo Horizonte
3

Now let's complicate our Json file a bit, let's put an Array of users and let's see how we should proceed to access the keys and values of this Json.

{
  "users":
  [
    {
    "name" : "Andre Almar",
    "type" : "knowledge",
    "age" : 25,
    "nick": "@andrealmar"
    },
    {
    "name" : "Jefferson",
    "type" : "knowledge",
    "age" : 160,
    "nick": "@jeffotoni"
    },
    {
    "name" : "Ieso Dias",
    "type" : "knowledge",
    "age" : 23,
    "nick": "@iesodias"
    }
  ]
}

Now our file is an Array of values, let's see how we do it to access the keys and values of our new Json.

Let's take a look at the code:

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"os"
	"reflect"
)

func main() {

	// We can use the ioutil.ReadFile which would already
	// return bytes and we would not need to use two functions
	// ioutil.ReadFile("./user-only.json")

	// Open our jsonFile
	jsonFile, err := os.Open("./user-only.json")

	// if we os.Open returns an error then handle it
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println("Successfully Opened users.json")

	// defer the closing of our jsonFile so that we can parse it later on
	defer jsonFile.Close()

	// reading archive content
	byteValue, _ := ioutil.ReadAll(jsonFile)

	// defining the interface map that will receive
	// the file and the format is dynamic
	var result map[string]interface{}

	// the syntax below is also allowed
	//var result = make(map[string]interface{})

	// let's now run our Unmarshal and convert to objects
	json.Unmarshal([]byte(byteValue), &result)

	// Let's access the interface on our map
	obj := result["users"]

	// Here completely changes from the
	// previous example we now have an [] interface,
	// ie an array of interfaces a slice.
	objInterface := obj.([]interface{})

	// Now as we have multiple users,
	// we'll loop through to fetch them
	// and access each user's key
	// and value in the slice
	for line, keyOne := range objInterface {

		fmt.Println("#### line ", line)

		// ValueOf returns a new Value initialized
		// to the concrete value stored in the interface i.
		val := reflect.ValueOf(keyOne)

		// A Kind represents the
		// specific kind of type
		// that a Type represents.
		if val.Kind() == reflect.Map {

			// Loop the map
			for _, key := range val.MapKeys() {

				// Index of Map
				v := val.MapIndex(key)

				// get the value of type Interface
				switch value := v.Interface().(type) {

				// v.Interface().(type)
				// here test the types
				case int:
					fmt.Println(key, value)
				case float64:
					fmt.Println(key, value)
				case string:
					fmt.Println(key, value)
				case bool:
					fmt.Println(key, value)
				default:
					fmt.Println("not found")
				}
			}
		}
	}
}

Output:

Successfully Opened users.json
k: 0 v: map[name:Andre Almar city:Sao Paulo age:25 nick:@andrealmar]
name Andre Almar
city Sao Paulo
age 25
nick @andrealmar
k: 1 v: map[name:Jefferson city:Belo Horizonte age:160 nick:@jeffotoni]
name Jefferson
city Belo Horizonte
age 160
nick @jeffotoni
k: 2 v: map[age:23 nick:@iesodias name:Ieso Dias city:Mato Grosso]
name Ieso Dias
city Mato Grosso
age 23
nick @iesodias

What if I have a Recursion in Json?

{
  "users":
  [
    {
      "name" : "Joelson",
      "city" : "Porto Alegre",
      "age" : 39,
      "social" : {
        "facebook" : "https://facebook.com/joelson",
        "twitter" : "https://twitter.com/joelson",
        "instagram" : "https://instagram.com/joelson"
      },
      "fone" : {
        "cell" : "5531987387246",
        "resid1" : "55314565678",
        "job" : "55314785679"
      }
    },
    {
      "name" : "Jefferson",
      "city" : "Belo Horizonte",
      "age" : 160,
      "social" : {
        "facebook" : "https://facebook.com/jeffotoni",
        "twitter" : "https://twitter.com/jeffotoni",
        "instagram" : "https://instagram.com/jeffotoni"
      },
      "fone" : {
        "cell" : "5531987387246",
        "resid1" : "55314565678",
        "job" : "55314785679"
      }
    }
  ]
}

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"reflect"
)

func main() {

	// Open our jsonFile
	byteJson, err := ioutil.ReadFile("./users.json")

	// if we os.Open returns an error then handle it
	if err != nil {
		fmt.Println(err)
		return
	}

	// show screen
	fmt.Println("Successfully Opened users.json")

	// defining the interface map that will receive
	// the file and the format is dynamic
	var result = make(map[string]interface{})

	// let's now run our Unmarshal and convert to objects
	json.Unmarshal(byteJson, &result)

	obj := result["users"]

	// get []interface
	objData := obj.([]interface{})

	// recursive
	DecodeMapVetInterface(objData)
}

func DecodeMapVetInterface(objData []interface{}) {
	//fmt.Println(objData)
	for line, v := range objData {

		fmt.Println("#### line: ", line)

		// ValueOf returns a new Value initialized
		// to the concrete value stored in the interface i.
		val := reflect.ValueOf(v)

		// A Kind represents the
		// specific kind of type
		// that a Type represents.
		if val.Kind() == reflect.Map {

			// Loop the map
			for _, key := range val.MapKeys() {

				fmt.Println("..............................................")

				// Index
				v := val.MapIndex(key)
				switch iv := v.Interface().(type) {

				// v.Interface().(type)
				// here test the types
				case int:
					fmt.Println(key, iv)
				case float64:
					fmt.Println(key, iv)
				case string:
					fmt.Println(key, iv)
				case bool:
					fmt.Println(key, iv)
				default:
					fmt.Println(key)
					DecodeMapInterface(iv)
				}
			}
		}
	}
}

func DecodeMapInterface(v interface{}) {
	val := reflect.ValueOf(v)
	if val.Kind() == reflect.Map {
		for _, e := range val.MapKeys() {
			v := val.MapIndex(e)
			switch t := v.Interface().(type) {
			case int:
				fmt.Println(e, t)
			case float64:
				fmt.Println(e, t)
			case string:
				fmt.Println(e, t)
			case bool:
				fmt.Println(e, t)
			default:
				fmt.Println("not found!")
			}
		}
	}
}

Output:

Successfully Opened users.json
#### line:  0
..............................................
name Joelson
..............................................
city Porto Alegre
..............................................
age 39
..............................................
social
facebook https://facebook.com/joelson
twitter https://twitter.com/joelson
instagram https://instagram.com/joelson
..............................................
fone
cell 5531987387246
resid1 55314565678
job 55314785679
#### line:  1
..............................................
name Jefferson
..............................................
city Belo Horizonte
..............................................
age 160
..............................................
social
instagram https://instagram.com/jeffotoni
facebook https://facebook.com/jeffotoni
twitter https://twitter.com/jeffotoni
..............................................
fone
cell 5531987387246
resid1 55314565678
job 55314785679

We were able to recursively list our Json, with several layers. But every Json file will have to be handled, structures change as needed, so this recursion will only work for this file template.

Let's take another example, with a new json with several layers.

Let's take a look at the code below:

{
"payload": 
  [
    {
        "products": {
            "house": {
                "rectangular bed": {
                    "price": 99.33,
                    "weight": 44
                },
                "Corner table": {
                    "price": 100.00,
                    "weight": 400
                }
            },
            "car": {
                "bmw": {
                    "price": 400.000,
                    "weight": 2000
                },
                "jaguar": {
                    "price": 600.000,
                    "weight": 3000
                }
            },
            "robotics": {
                "arduino circuit board": {
                    "price": 3000,
                    "weight": 140
                },
                "proximity sensor...": {
                    "price": 100,
                    "weight": 34
                },
                "temperature sensor": {
                    "price": 344,
                    "weight": 55
                }
            }
        }
    }
  ]
}

Complete code:

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
)

func main() {

	// open file json
	data, err := ioutil.ReadFile("./payload.json")
	if err != nil {
		fmt.Println(err)
		return
	}

	// make map interface
	payload := make(map[string]interface{})

	// Unmarshal data
	err = json.Unmarshal(data, &payload)
	if err != nil {
		fmt.Println(err)
		return
	}

	// call recursive
	recursivMap(payload)
}

// recursive Map
func recursivMap(payload map[string]interface{}) {
	for k, v := range payload {
		//fmt.Printf("%v %T: %v\n", k, v, v)
		switch v.(type) {
		case []interface{}:
			recursivSlice(v.([]interface{}))
		case map[string]interface{}:
			//fmt.Printf("%v %T: %v\n", k, v, v)
			fmt.Println("----------------------")
			fmt.Printf("%v\n", k)
			recursivMap(v.(map[string]interface{}))

		default:
			fmt.Printf("%v=%v\n", k, v)
		}
	}
}

// recursive Slice
func recursivSlice(pauload []interface{}) {
	for _, v := range pauload {
		switch v.(type) {
		case []interface{}:
			recursivSlice(v.([]interface{}))
		case map[string]interface{}:
			recursivMap(v.(map[string]interface{}))
		}
	}
}

Output:

----------------------
products
----------------------
house
----------------------
rectangular bed
price=99.33
weight=44
----------------------
Corner table
price=100
weight=400
----------------------
car
----------------------
bmw
price=400
weight=2000
----------------------
jaguar
price=600
weight=3000
----------------------
robotics
----------------------
arduino circuit board
price=3000
weight=140
----------------------
proximity sensor...
price=100
weight=34
----------------------
temperature sensor
price=344
weight=55

Now, we did something totally recursive, presenting all the keys and values from our Json file.

Parsing in yaml Format Using Go

After this bunch of information we learn how to do parse with reflection using Go we are ready to create our own libs to parse in files. Reflection is very powerful and has several features and applicabilities, existing libs use reflection to parse files in json, toml, yaml or gcfg formats.

Let's see in practice the lib gopkg.in/yaml.v2, and let's parse a file in Yaml format.

We have to install the lib on our local machine and to do this just use the command below.

$ go get -u gopkg.in/yaml.v2

Check out the complete code below

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"os"

	yaml "gopkg.in/yaml.v2"
)

type ServersSsh struct {
	Version string `yaml:"version"`

	Info struct {
		Description string `yaml:"description"`
	}

	Server1 struct {
		Host    string `yaml:"host"`
		Port    string `yaml:"port"`
		User    string `yaml:"user"`
		FilePem string `yaml:"filepem"`
		KeyAws  string `yaml:"keyaws"`
	}

	Server2 struct {
		Host    string `yaml:"host"`
		Port    string `yaml:"port"`
		User    string `yaml:"user"`
		FilePem string `yaml:"filepem"`
		KeyAws  string `yaml:"keyaws"`
	}
}

// Our config case has no structure created
// the system will dynamically create
var YamlMemory = `
`

// func main
func main() {

	var yamlByte []byte
	var Yaml ServersSsh
	var err error

	// local
	file := "server.yaml"

	// test exist
	if !FileExist(file) {
		fmt.Println(file + " not exist!")
		return
	}

	if yamlByte, err = ioutil.ReadFile(file); err != nil {
		log.Println("Error: ", err)
	}

	// Unmarshal receives the file in a byte format and assigns the values passed to the fields in the structure.
	// If there is an error it displays an error message on the screen informing the error
	if err := yaml.Unmarshal(yamlByte, &Yaml); err != nil {
		log.Println("Error", err)
	}

	fmt.Println("Version: ", Yaml.Version)
	fmt.Println("Description: ", Yaml.Info.Description)
	fmt.Println("Server1.Host: ", Yaml.Server1.Host)
	fmt.Println("Server2.Host: ", Yaml.Server2.Host)
}

func FileExist(name string) bool {
	//if _, err := os.Stat(name); os.IsNotExist(err) {
	if stat, err := os.Stat(name); err == nil && !stat.IsDir() {
		return true
	}
	return false
}

Output:

Version:  1.0
Description:  this is an example of yaml file to server an ssh server or multiple servers to do ssh automating services on server
Server1.Host:  127.0.0.1
Server2.Host:  127.0.0.10

Parsing in Toml Format Using Go

Now let's parse Toml files, there are several libs that do this, we will use github.com/BurntSushi/toml to do our parses.

The cool thing is that we have seen what the libs are doing behind the scenes, now to use the libs is very easy and all our understanding of struct, maps helped to unmask what happens behind the scenes.

We will install the package so that everything goes right.

$ go get github.com/BurntSushi/toml

Try the toml validator:

$ go get github.com/BurntSushi/toml/cmd/tomlv
$ tomlv server.toml

Let's face it, check out the full code below.

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/BurntSushi/toml"
)

type ConfToml struct {
	Version string
	Info    Info
	Server1 Server1
	Server2 Server2
	Clients Clients
}

// Info from config file
type Info struct {
	Description string
}

// Server1
type Server1 struct {
	Host    string
	Port    int64
	User    string
	FilePem string
	KeyAws  string
}

// Server2
type Server2 struct {
	Host    string
	Port    int64
	User    string
	FilePem string
	KeyAws  string
}

// info from clients file
type Clients struct {
	Ping  string
	Data  [][]interface{} // very beautiful
	Hosts []string
}

func main() {

	var Toml ConfToml

	file := "server.toml"

	if !FileExist(file) {
		fmt.Println(file + " not exist!")
		return
	}

	if _, err := toml.DecodeFile(file, &Toml); err != nil {
		log.Fatal(err)
	}

	// config data
	fmt.Println("version:", Toml.Version)
	fmt.Println("Info.Description: ", Toml.Info.Description)
	fmt.Println("Server1.Host: ", Toml.Server1.Host)
	fmt.Println("Server2.Host:", Toml.Server2.Host)

	fmt.Println(Toml.Clients.Data)
	fmt.Println(Toml.Clients.Data[0])
	fmt.Println(Toml.Clients.Data[0][0])
	fmt.Println(Toml.Clients.Data[0][1])

	fmt.Println(Toml.Clients.Hosts)
	fmt.Println(Toml.Clients.Hosts[0])
	fmt.Println(Toml.Clients.Hosts[1])
}

func FileExist(name string) bool {
	//if _, err := os.Stat(name); os.IsNotExist(err) {
	if stat, err := os.Stat(name); err == nil && !stat.IsDir() {
		return true
	}
	return false
}

Output:

version: 1.0
Info.Description:  this is an example of yaml file to server an ssh server or multiple servers to do ssh automating services on server
Server1.Host:  127.0.0.1
Server2.Host: 127.0.0.10
[[amazon google] [1 2]]
[amazon google]
amazon
google
[aws cloud]
aws
cloud

Parsing with Viper

When building a modern application, you don’t want to worry about configuration file formats; you want to focus on building awesome software. Viper is here to help with that.

Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed to work within an application, and can handle all types of configuration needs and formats. It supports:

 - setting defaults
 - reading from JSON, TOML, YAML, HCL, and Java properties config files
 - live watching and re-reading of config files (optional)
 - reading from environment variables
 - reading from remote config systems (etcd or Consul), and watching changes
 - reading from command line flags
 - reading from buffer
 - setting explicit values

Viper can be thought of as a registry for all of your applications configuration needs.

Let's now see some practical examples of how to work with viper.

Viper file Json, Yaml and Toml

$ go get github.com/spf13/viper

Look at the code below:

package main

import (
	"fmt"

	"github.com/spf13/viper"
)

func main() {

	// viper.SetConfigType("toml")
	// viper.SetConfigName("servert") // name of config file (without extension)

	// viper.SetConfigType("yaml")
	// viper.SetConfigName("servery") // name of config file (without extension)

	viper.SetConfigType("json")
	viper.SetConfigName("serverj") // name of config file (without extension)

	viper.AddConfigPath(".") // optionally look for config in the working directory

	//viper.AddConfigPath("/etc/appname/")  // path to look for the config file in
	//viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths

	err := viper.ReadInConfig() // Find and read the config file
	if err != nil {             // Handle errors reading the config file
		panic(fmt.Errorf("Fatal error config file: %s \n", err))
	}

	viper.Set("Verbose", true)
	//viper.Set("LogFile", LogFile)

	fmt.Println("datastore.metric.host: ", viper.GetString("datastore.metric.host"))
	fmt.Println("datastore.warehouse.host: ", viper.GetString("datastore.warehouse.host"))

	fmt.Println("version.json: ", viper.GetString("version"))
	fmt.Println("info.json: ", viper.GetString("info.description"))
	fmt.Println("server1.host.json: ", viper.GetString("server1.host"))
	fmt.Println("server1.user.json: ", viper.GetString("server1.user"))

	fmt.Println("server2.host.json: ", viper.GetString("server2.host"))
	fmt.Println("server2.user.json: ", viper.GetString("server2.user"))
}

Output:

datastore.metric.host:  192.168.0.113
datastore.warehouse.host:  198.0.0.1
version.json:  1.0
info.json:  this is an example of yaml file to server an ssh ..
server1.host.json:  127.0.0.1
server1.user.json:  ubuntu
server2.host.json:  127.0.0.13
server2.user.json:  centos

In the above example the great advantage that we did not change our code to adapt to any specific pattern, we read Json, Toml and Yaml without having to change a line.

One realizes that we did not create struct to capture the fields nor to create collections, the viper did everything for people, all magic that we learned of reflection he applied in his lib.

Viper Map with Interface

We made an interface map to test with Viper.

From a check how it was:

package main

import (
	"fmt"

	"github.com/spf13/viper"
)

func main() {
	v1, err := readConfig("enviroment", map[string]interface{}{
		"version": 1.0,
		"info": map[string]string{
			"description": "this is an example of yaml file to server an ssh server or",
		},
		"server1": map[string]string{
			"host": "localhost",
			"user": "titpetric",
		},
	})
	if err != nil {
		panic(fmt.Errorf("Error when reading config: %v\n", err))
	}

	version := v1.GetFloat64("version")
	info := v1.GetString("info.description")
	server1 := v1.GetStringMapString("server1")

	fmt.Printf("version: %0.1f\n", version)
	fmt.Printf("info.description: %s\n", info)
	fmt.Printf("server1:%#v\n", server1)
	fmt.Printf("server1.host: %s\n", server1["host"])
}

func readConfig(filename string, defaults map[string]interface{}) (*viper.Viper, error) {
	v := viper.New()
	for key, value := range defaults {
		v.SetDefault(key, value)
	}

	v.SetConfigName(filename)
	v.AddConfigPath(".")
	v.AutomaticEnv()
	err := v.ReadInConfig()
	return v, err
}

version: 1.0
info.description: this is an example of yaml file to server an ssh server or multiple servers to do ssh automating services on server
server1:map[string]string{"type":"env", "file":"key_1.pem", "env":"KEY_AWS", "host":"127.0.0.1", "port":"22", "user":"ubuntu"}
server1.host: 127.0.0.1

Viper in Memory

The example below shows that we can use variables with the content in Yaml, Json or Toml format so that we can parse.

Check out the complete code below:

package main

import (
	"bytes"
	"fmt"

	"github.com/spf13/viper"
)

func main() {

	viper.SetConfigType("yaml") // or viper.SetConfigType("YAML")

	// any approach to require this configuration into your program.
	var yamlExample = []byte(`
version: 1.0
info: 
 description: this is an example of yaml file to ...
server1:
  host: 127.0.0.1
  port: 22
  user: ubuntu
  type: env
  file: key_1.pem
  env: KEY_AWS
  Clouds: [Aws, Google, Azure]
`)

	viper.ReadConfig(bytes.NewBuffer(yamlExample))

	fmt.Println(viper.Get("version"))
	fmt.Println(viper.Get("info.description"))
	fmt.Println(viper.Get("server1.host"))
	fmt.Println(viper.Get("server1.user"))
	mapsc := viper.GetStringSlice("server1.clouds")

	fmt.Println(mapsc[0])
}

1
this is an example of yaml file to ...
127.0.0.1
ubuntu
Aws

The viper is very powerful we can do Marshal, Unmarsha works with multiple viper, access, works with Flags etc.

It is worth checking.

Links Json to Golang

Below I am making available some links to convert from Json to Struct in Golang, it gets a json or you write your Json and it mounts the struct for you. Of course it helps when you know what you're doing, and it's very useful sometimes to find some more complex json.

Lab 04 Building apis with net/http

Introduction http

Now we get to the best part, put into practice everything we learn. Let's get to know the net / http package one of the most powerful packages in Go, there are many speculations about it but we will really do our best in what it provides with the features it offers. There are many implementations on the net / http, several routes, frameworks, libs all to minimize the work and speed up various tasks when coding our apis. Our goal is to create native APIs, as we did in Lab 03 Parse with Golang.

Everything in Go follows this model, has lib for a lot, and the more you master the language the more habit you will be to choose the libs better or develop your own libs. Let's start by developing our API Server, so we can consume it later. APIS as a server can be done in several ways, either by building APIS in the rEST, GraphQL, SOAP, XML-RPC and several other forms of communication such as RPC, Socket or Websocket.

We have a powerful and vast library, everything we had in C or C ++ is in Go improved. Every net/http package is working on Goroutine, this is one of the pillars of net/http

type Handler

A Handler responds to an HTTP request.

ServeHTTP should write reply headers and data to the ResponseWriter and then return. Returning signals that the request is finished; it is not valid to use the ResponseWriter or read from the Request.Body after or concurrently with the completion of the ServeHTTP call.

Once implemented the http.Handler can be passed to http.ListenAndServe, which will call the ServeHTTP method on every incoming request. http.Request contains all relevant information about an incoming http request which is being served by your http.Handler.

type Handler interface {
        ServeHTTP(ResponseWriter, *Request)
}

Check the code below:

package main

import (
	"fmt"
	"log"
	"net/http"
)

type pingHandler struct{}

func (h pingHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "DevopsBH for Golang simple %s\n", r.URL.Path)
}

func main() {
	log.Printf("\nServer run 8080\n")
	err := http.ListenAndServe(":8080", pingHandler{})
	log.Fatal(err)
}

Run the curl:

$ curl -i -Xget localhost:8080/v1/api/ping
$ curl -i -Xget localhost:8080

The http.ResponseWriter is the interface through which you can respond to the request. It implements the io.Writer interface, so you can use methods like fmt.Fprintf to write a formatted string as the response body, or ones like io.Copy to write out the contents of a file (or any other io.Reader). The response code can be set before you begin writing data using the WriteHeader method.

Go’s http package has turned into one of my favorite things about the Go programming language. Initially it appears to be somewhat complex, but in reality it can be broken down into a couple of simple components that are extremely flexible in how they can be used.

Type Handlerfunc

The HandlerFunc type is an adapter to allow the use of ordinary functions as HTTP handlers. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler that calls f.

Often defining a full type to implement the http.Handler interface is a bit overkill, especially for extremely simple ServeHTTP functions like the one above. The http package provides a helper function, http.HandlerFunc, which wraps a function which has the signature func(w http.ResponseWriter, r http.Request), returning an http.Handler which will call it in all cases.

The following behaves exactly like the previous example, but uses http.HandlerFunc instead of defining a new type.

type HandlerFunc func(ResponseWriter, *Request)

Check out:

handlerApiPing := http.HandlerFunc(Ping)

Look at the code below:

package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	handlerfunc := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "DevopsBH for Golang simple two %s\n", r.URL.Path)
	})

	log.Printf("\nServer run 8080\n")
	err := http.ListenAndServe(":8080", handlerfunc)
	log.Fatal(err)
}

Func http Handlefunc

HandleFunc registers the handler function for the given pattern in the DefaultServeMux. The documentation for ServeMux explains how patterns are matched.

func HandleFunc(pattern string, handler func(ResponseWriter, *Request))

Check out the examples below:

http.HandleFunc("/v1/api/ping", pingHandler)

http.HandleFunc("/v1/api/ping", func(w http.ResponseWriter, req *http.Request){})

package main

import (
	"log"
	"net/http"
)

func main() {

	// our function
	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		w.Write([]byte("\nDevops BH for Golang HandleFunc!"))
	}

	// handleFunc
	http.HandleFunc("/v1/api/ping", pingHandler)
	http.HandleFunc("/v1/api/ping2", pingHandler)
	http.HandleFunc("/v1/api/ping3", pingHandler)

	// show run server
	log.Printf("\nServer run 8080\n")

	// Listen
	log.Fatal(http.ListenAndServe(":8080", nil))
}

Func http Handle

Handle registers the handler for the given pattern in the DefaultServeMux.

func Handle(pattern string, handler Handler)

Check out the example below:

http.Handle("/v1/api/ping", http.HandlerFunc(Ping))

package main

import (
	"log"
	"net/http"
)

func main() {

	// our function
	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		w.Write([]byte("\nDevops BH for Golang Handle tree!"))
	}

	// Handle and recive http.HandlerFunc
	http.Handle("/v1/api/ping", http.HandlerFunc(pingHandler))
	http.Handle("/v1/api/ping2", http.HandlerFunc(pingHandler))
	http.Handle("/v1/api/ping3", http.HandlerFunc(pingHandler))
	// http.Handle("/v1/api/ping", pingHandler) // error

	// show run
	log.Printf("\nServer run 8080\n")

	// Listen
	// log.Fatal(http.ListenAndServe(":8080", http.HandlerFunc(pingHandler))) ok
	log.Fatal(http.ListenAndServe(":8080", nil))
}

Func http Error

Error replies to the request with the specified error message and HTTP code. It does not otherwise end the request; the caller should ensure no further writes are done to w. The error message should be plain text.

func Error(w ResponseWriter, error string, code int)

Check out the example below:

json := `{"status":"error", "msg":"method not supported, only POST"}`
http.Error(w, json, http.StatusUnauthorized)

Constants Common HTTP Methods

Unless otherwise noted, these are defined in RFC 7231 section 4.3.

 const (
        MethodGet     = "GET"
        MethodHead    = "HEAD"
        MethodPost    = "POST"
        MethodPut     = "PUT"
        MethodPatch   = "PATCH" // RFC 5789
        MethodDelete  = "DELETE"
        MethodConnect = "CONNECT"
        MethodOptions = "OPTIONS"
        MethodTrace   = "TRACE"
)

HTTP status codes as registered with IANA. See: http/status/code

const (
        StatusContinue           = 100 // RFC 7231, 6.2.1
        StatusSwitchingProtocols = 101 // RFC 7231, 6.2.2
        StatusProcessing         = 102 // RFC 2518, 10.1

        StatusOK                   = 200 // RFC 7231, 6.3.1
        StatusCreated              = 201 // RFC 7231, 6.3.2
        StatusAccepted             = 202 // RFC 7231, 6.3.3
        StatusNonAuthoritativeInfo = 203 // RFC 7231, 6.3.4
        StatusNoContent            = 204 // RFC 7231, 6.3.5
        StatusResetContent         = 205 // RFC 7231, 6.3.6
        StatusPartialContent       = 206 // RFC 7233, 4.1
        StatusMultiStatus          = 207 // RFC 4918, 11.1
        StatusAlreadyReported      = 208 // RFC 5842, 7.1
        StatusIMUsed               = 226 // RFC 3229, 10.4.1

        StatusMultipleChoices  = 300 // RFC 7231, 6.4.1
        StatusMovedPermanently = 301 // RFC 7231, 6.4.2
        StatusFound            = 302 // RFC 7231, 6.4.3
        StatusSeeOther         = 303 // RFC 7231, 6.4.4
        StatusNotModified      = 304 // RFC 7232, 4.1
        StatusUseProxy         = 305 // RFC 7231, 6.4.5

        StatusTemporaryRedirect = 307 // RFC 7231, 6.4.7
        StatusPermanentRedirect = 308 // RFC 7538, 3

        StatusBadRequest                   = 400 // RFC 7231, 6.5.1
        StatusUnauthorized                 = 401 // RFC 7235, 3.1
        StatusPaymentRequired              = 402 // RFC 7231, 6.5.2
        StatusForbidden                    = 403 // RFC 7231, 6.5.3
        StatusNotFound                     = 404 // RFC 7231, 6.5.4
        StatusMethodNotAllowed             = 405 // RFC 7231, 6.5.5
        StatusNotAcceptable                = 406 // RFC 7231, 6.5.6
        StatusProxyAuthRequired            = 407 // RFC 7235, 3.2
        StatusRequestTimeout               = 408 // RFC 7231, 6.5.7
        StatusConflict                     = 409 // RFC 7231, 6.5.8
        StatusGone                         = 410 // RFC 7231, 6.5.9
        StatusLengthRequired               = 411 // RFC 7231, 6.5.10
        StatusPreconditionFailed           = 412 // RFC 7232, 4.2
        StatusRequestEntityTooLarge        = 413 // RFC 7231, 6.5.11
        StatusRequestURITooLong            = 414 // RFC 7231, 6.5.12
        StatusUnsupportedMediaType         = 415 // RFC 7231, 6.5.13
        StatusRequestedRangeNotSatisfiable = 416 // RFC 7233, 4.4
        StatusExpectationFailed            = 417 // RFC 7231, 6.5.14
        StatusTeapot                       = 418 // RFC 7168, 2.3.3
        StatusMisdirectedRequest           = 421 // RFC 7540, 9.1.2
        StatusUnprocessableEntity          = 422 // RFC 4918, 11.2
        StatusLocked                       = 423 // RFC 4918, 11.3
        StatusFailedDependency             = 424 // RFC 4918, 11.4
        StatusUpgradeRequired              = 426 // RFC 7231, 6.5.15
        StatusPreconditionRequired         = 428 // RFC 6585, 3
        StatusTooManyRequests              = 429 // RFC 6585, 4
        StatusRequestHeaderFieldsTooLarge  = 431 // RFC 6585, 5
        StatusUnavailableForLegalReasons   = 451 // RFC 7725, 3

        StatusInternalServerError           = 500 // RFC 7231, 6.6.1
        StatusNotImplemented                = 501 // RFC 7231, 6.6.2
        StatusBadGateway                    = 502 // RFC 7231, 6.6.3
        StatusServiceUnavailable            = 503 // RFC 7231, 6.6.4
        StatusGatewayTimeout                = 504 // RFC 7231, 6.6.5
        StatusHTTPVersionNotSupported       = 505 // RFC 7231, 6.6.6
        StatusVariantAlsoNegotiates         = 506 // RFC 2295, 8.1
        StatusInsufficientStorage           = 507 // RFC 4918, 11.5
        StatusLoopDetected                  = 508 // RFC 5842, 7.2
        StatusNotExtended                   = 510 // RFC 2774, 7
        StatusNetworkAuthenticationRequired = 511 // RFC 6585, 6
)

DefaultMaxHeaderBytes is the maximum permitted size of the headers in an HTTP request. This can be overridden by setting Server.MaxHeaderBytes.

const DefaultMaxHeaderBytes = 1 << 20 // 1 MB

DefaultMaxIdleConnsPerHost is the default value of Transport's MaxIdleConnsPerHost.

const DefaultMaxIdleConnsPerHost = 2

TimeFormat is the time format to use when generating times in HTTP headers. It is like time.RFC1123 but hard-codes GMT as the time zone. The time being formatted must be in UTC for Format to generate the correct format.

For parsing this time format, see ParseTime.

const TimeFormat = "Mon, 02 Jan 2006 15:04:05 GMT"

Type ServeMux

ServeMux is an HTTP request multiplexer. It matches the URL of each incoming request against a list of registered patterns and calls the handler for the pattern that most closely matches the URL.

Patterns name fixed, rooted paths, like "/favicon.ico", or rooted subtrees, like "/images/" (note the trailing slash). Longer patterns take precedence over shorter ones, so that if there are handlers registered for both "/images/" and "/images/thumbnails/", the latter handler will be called for paths beginning "/images/thumbnails/" and the former will receive requests for any other paths in the "/images/" subtree.

Note that since a pattern ending in a slash names a rooted subtree, the pattern "/" matches all paths not matched by other registered patterns, not just the URL with Path == "/".

If a subtree has been registered and a request is received naming the subtree root without its trailing slash, ServeMux redirects that request to the subtree root (adding the trailing slash). This behavior can be overridden with a separate registration for the path without the trailing slash. For example, registering "/images/" causes ServeMux to redirect a request for "/images" to "/images/", unless "/images" has been registered separately.

Patterns may optionally begin with a host name, restricting matches to URLs on that host only. Host-specific patterns take precedence over general patterns, so that a handler might register for the two patterns "/codesearch" and "codesearch.google.com/" without also taking over requests for "http://www.google.com/".

ServeMux also takes care of sanitizing the URL request path and the Host header, stripping the port number and redirecting any request containing . or .. elements or repeated slashes to an equivalent, cleaner URL.

type ServeMux struct {
     // contains filtered or unexported fields
}

Type NewServeMux

NewServeMux allocates and returns a new ServeMux.

func NewServeMux() *ServeMux

Check out:

mux := http.NewServeMux()

func (*ServeMux) HandleFunc

func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request))

HandleFunc registers the handler function for the given pattern.

Check the code below:

package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {

	mux := http.NewServeMux()

	// our function
	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		w.Write([]byte("\nDevops BH for Golang mux HandleFunc!"))
	}

	// handleFunc
	mux.HandleFunc("/v1/api/ping", pingHandler) // ok

	mux.HandleFunc("/v1/api/ping2", http.HandlerFunc(pingHandler)) // ok

	mux.HandleFunc("/v1/api/ping3", pingHandler) // ok

	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusNotFound)
		fmt.Fprintln(w, "You're lost, go home devopsBH!")
	})

	log.Printf("\nServer run 8080\n")
	// Listen
	log.Fatal(http.ListenAndServe(":8080", mux))
}

Run cURL:

$ curl -i -Xget localhost:8080/

Type ServeMux Handle

Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.

func (mux *ServeMux) Handle(pattern string, handler Handler)

Check out:

mux := http.NewServeMux()
mux.Handle("/v1/api/ping", http.HandlerFunc(Ping))

package main

import (
	"log"
	"net/http"
)

func main() {

	mux := http.NewServeMux()

	// our function
	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		w.Write([]byte("\nDevops BH for Golang mux Handle()!"))
	}

	// handlerFunc
	mux.Handle("/v1/api/ping", http.HandlerFunc(pingHandler)) // ok
	// mux.Handle("/v1/api/ping2", pingHandler) // error
	// mux.Handle("/v1/api/ping", mux.HandlerFunc(pingHandler)) // error

	// mux.Handle("/", func(w http.ResponseWriter, r *http.Request) {  // error
	// 	w.WriteHeader(http.StatusNotFound)
	// 	fmt.Fprintln(w, "You're lost, go home devopsBH!")
	// })

	log.Printf("\nServer run 8080\n")
	// Listen
	log.Fatal(http.ListenAndServe(":8080", mux))
}

Run cURL:

$ curl -i -Xget localhost:8080/

Func ListenAndServe

HandleFunc registers the handler function for the given pattern in the DefaultServeMux. The documentation for ServeMux explains how patterns are matched.

func ListenAndServe(addr string, handler Handler) error

ListenAndServe listens on the TCP network address addr and then calls Serve with handler to handle requests on incoming connections. Accepted connections are configured to enable TCP keep-alives.

The handler is typically nil, in which case the DefaultServeMux is used.

ListenAndServe always returns a non-nil error.

Check out our first example:

package main

import (
	"io"
	"log"
	"net/http"
)

func main() {

	// our function
	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		io.WriteString(w, "DevopsBH, Golang for Devops!\n")
	}

	// handlerFunc
	http.HandleFunc("/v1/api/ping", pingHandler)

	// Listen
	// log.Fatal(http.ListenAndServe(":8080", http.HandlerFunc(pingHandler))) ok
	log.Fatal(http.ListenAndServe(":8080", nil)) // ok

}

In this apis scenario, the program is listening on the port determined by the function ListenAndServe waiting for the requests to be received so that it can respond to incoming requests.

$ curl -i -XPOST localhost:8080/v1/api/ping

Output:

HTTP/1.1 200 OK
Date: Fri, 01 Feb 2019 17:01:23 GMT
Content-Length: 29
Content-Type: text/plain; charset=utf-8

DevopsBH, Golang for Devops!

func ListenAndServeTLS

ListenAndServeTLS acts identically to ListenAndServe, except that it expects HTTPS connections. Additionally, files containing a certificate and matching private key for the server must be provided. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server's certificate, any intermediates, and the CA's certificate.

func ListenAndServeTLS(addr, certFile, keyFile string, handler Handler) error

Before we have to generate the keys, .pem or .crt and the .key file. Let's generate all running openssl.

Check the codes below:

#generating .key and .csr
$ openssl req -nodes -newkey rsa:2048 -keyout server.key -out server.csr -subj "/C=BR/ST=Minas/L=Belo Horizonte/O=s3wf Ltd./OU=IT/CN=localhost"

# generating server .crt or .pem
$ openssl x509 -req -sha256 -in server.csr -signkey server.key -out server.crt -days 365

Soon we generate server.crt, server.csr, server.key.

Now, let's go to our api below:

package main

import (
	"io"
	"log"
	"net/http"
)

var (
	addr = ":443"
)

func main() {

	http.HandleFunc("/v1/api/ping", func(w http.ResponseWriter, req *http.Request) {
		io.WriteString(w, "DevopsBH, Golang for Devops TLS!\n")
	})

	// show
	log.Printf("Server Run %s TLS / https://localhost%s", addr, addr)

	// conf listen TLS
	err := http.ListenAndServeTLS(addr, "server.crt", "server.key", nil)
	log.Fatal(err)
}

Below the same code however modifying listen TLS using http.HandlerFunc()

Check the code below:

package main

import (
	"io"
	"log"
	"net/http"
)

var (
	addr = ":443"
)

func main() {

	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		io.WriteString(w, "DevopsBH, Golang for Devops TLS!\n")
	}

	// show
	log.Printf("Server Run %s TLS / https://localhost%s", addr, addr)

	// conf listen TLS
	err := http.ListenAndServeTLS(addr, "server.crt", "server.key", http.HandlerFunc(pingHandler))
	log.Fatal(err)
}

// curl --insecure -i -XGET https://localhost:8443/v1/api/ping
// curl -k -i -XGET https://localhost:8443/v1/api/ping

$ curl --insecure -i -XGET https://localhost:443/v1/api/ping
or
$ curl -k -i -XGET https://localhost:443/v1/api/ping

Now we will use some properties of the tls package and we will make a config, as we have already learned mux we will use it as well. At first it seems confusing, but in fact it's simple let's check it out.

Look at the code below:

package main

import (
	"crypto/tls"
	"io"
	"log"
	"net/http"
)

var (
	addr = ":443"
)

func main() {

	mux := http.NewServeMux()

	mux.HandleFunc("/v1/api/ping",
		func(w http.ResponseWriter, req *http.Request) {
			w.Header().Add("Strict-Transport-Security", "max-age=63072000; includeSubDomains")
			io.WriteString(w, "DevopsBH, Golang for Devops TLS MUX!\n")
		})

	cfg := &tls.Config{
		MinVersion:               tls.VersionTLS12,
		CurvePreferences:         []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256},
		PreferServerCipherSuites: true,
		CipherSuites: []uint16{
			tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
			tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
			tls.TLS_RSA_WITH_AES_256_GCM_SHA384,
			tls.TLS_RSA_WITH_AES_256_CBC_SHA,
		},
	}

	srv := &http.Server{
		Addr:         addr,
		Handler:      mux,
		TLSConfig:    cfg,
		TLSNextProto: make(map[string]func(*http.Server, *tls.Conn, http.Handler), 0),
	}

	// show
	log.Printf("Server Run %s TLS / https://localhost%s", addr, addr)

	// conf listen TLS
	err := srv.ListenAndServeTLS("server.crt", "server.key")
	log.Fatal(err)
}

// curl --insecure -i -XGET https://localhost:443/v1/api/ping
// curl -k -i -XGET https://localhost:443/v1/api/ping

Run cURL:

$ curl --insecure -i -XGET https://localhost:443/v1/api/ping
or
$ curl -k -i -XGET https://localhost:443/v1/api/ping

Now let's put some functions that will make a difference when we run our api for high performance, let's try not to use the fmt library to write to the monitor, let's use io and buff. Well performance is something absurdly faster.

From one checked in the complete code below:

package main

import (
	"bufio"
	"io"
	"log"
	"net/http"
	"os"
)

var (
	addr = ":8080"
)

// write bufio to optimization
func write(text string) {
	// var writer *bufio.Writer
	writer := bufio.NewWriter(os.Stdout)
	writer.WriteString(text)
	writer.Flush()
}

func main() {

	// our function
	pingHandler := func(w http.ResponseWriter, req *http.Request) {
		json := `{"status":"success", "msg":"DevopsBH, Golang for Devops!"}`
		w.Header().Set("Content-Type", "application/json; charset=utf-8")
		w.WriteHeader(http.StatusUnauthorized)
		io.WriteString(w, json)
	}

	// handlerFunc
	http.HandleFunc("/v1/api/ping", pingHandler)

	// show
	write("\033[0;33mServer Run Port " + addr + "\033[0m\n")

	// Listen
	log.Fatal(http.ListenAndServe(addr, nil))
}

// Go in action
// @jeffotoni
// 2019-01-01

package main

import (
	"bufio"
	"io"
	"log"
	"net/http"
	"os"
	"strings"
	"time"
)

var (
	addr = ":8080"
)

// show log on screen
func logf(method, uri, nameHandle string, timeHandler time.Duration) {

	expre := "\033[5m%s \033[0;103m%s\033[0m \033[0;93m%s\033[0m \033[1;44m%s\033[0m"
	log.Printf(expre, method, uri, nameHandle, timeHandler)
}

// write bufio to optimization
func write(text string) {
	// var writer *bufio.Writer
	writer := bufio.NewWriter(os.Stdout)
	writer.WriteString(text)
	writer.Flush()
}

func Ping(w http.ResponseWriter, r *http.Request) {

	// start time
	start := time.Now()

	if http.MethodPost == strings.ToUpper(r.Method) {

		json := `{"status":"success", "msg":"DevopsBH, Golang for Devops!"}`
		w.Header().Set("Content-Type", "application/json; charset=utf-8")
		w.WriteHeader(http.StatusOK)
		io.WriteString(w, json)

	} else {

		json := `{"status":"error", "msg":"method not supported, only POST"}`
		w.Header().Set("Content-Type", "application/json; charset=utf-8")
		w.WriteHeader(http.StatusUnauthorized)
		io.WriteString(w, json)
	}

	logf(r.Method,
		r.RequestURI,
		"Ping",
		time.Since(start))
}

func main() {

	// handlerFunc
	http.HandleFunc("/v1/api/ping", Ping)

	// show
	write("\033[0;33mServer Run " +
		"Port " +
		addr + "\033[0m\n")

	// Listen
	log.Fatal(http.ListenAndServe(addr, nil))
}

$ curl -i -XPOSt localhost:8080/v1/api/ping
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 01 Feb 2019 22:04:57 GMT
Content-Length: 58
{"status":"success", "msg":"DevopsBH, Golang for Devops!"}

$ curl -i -XGET localhost:8080/v1/api/ping
HTTP/1.1 401 Unauthorized
Content-Type: application/json; charset=utf-8
Date: Fri, 01 Feb 2019 22:05:46 GMT
Content-Length: 59
{"status":"error", "msg":"method not supported, only POST"}

Other Muxes

There are numerous replacements for http.ServeMux like gorilla/mux which give you things like automatically pulling variables out of paths, easily asserting what http methods are allowed on an endpoint, and more. Most of these replacements will implement http.Handler like http.ServeMux does, and accept http.Handlers as arguments, and so are easy to use in conjunction with the rest of the things I’m going to talk about in this post.

Let's write our own http.HandlerFunc, let's create something simple just so we can understand what happens with our apis.

Check the code below:

package main

import (
	"fmt"
	"log"
	"net/http"
)

type numberDumperString string
type numberDumperInt int

// http HandlerFunc
func (n numberDumperString) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "DevOps BH, Golang is Life, Here's your number: %s\n", n)
}

// http HandlerFunc
func (n numberDumperInt) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "DevOps BH, Golang is Life, Here's your number: %d\n", n)
}

func main() {
	mux := http.NewServeMux()

	mux.Handle("/one", numberDumperString("one"))
	mux.Handle("/two", numberDumperString("two"))
	mux.Handle("/three", numberDumperInt(3))
	mux.Handle("/four", numberDumperInt(4))
	mux.Handle("/five", numberDumperInt(5))

	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(404)
		fmt.Fprintln(w, "That's not a supported number new Ghoper!")
	})

	// show run
	log.Printf("\nServer run 8080\n")

	// listen
	err := http.ListenAndServe(":8080", mux)
	log.Fatal(err)
}

Run cURL:

$ curl -i -Xget localhost:8080/one
HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 01:25:39 GMT
Content-Length: 51
Content-Type: text/plain; charset=utf-8

DevOps BH, Golang is Life, Here's your number: one

$ curl -i -Xget localhost:8080/two 
HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 01:25:39 GMT
Content-Length: 51
Content-Type: text/plain; charset=utf-8

DevOps BH, Golang is Life, Here's your number: two

$ curl -i -Xget localhost:8080/three
HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 01:25:39 GMT
Content-Length: 51
Content-Type: text/plain; charset=utf-8

DevOps BH, Golang is Life, Here's your number: three

$ curl -i -Xget localhost:8080/four
HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 01:25:39 GMT
Content-Length: 51
Content-Type: text/plain; charset=utf-8

DevOps BH, Golang is Life, Here's your number: four

$ curl -i -Xget localhost:8080/eleven

HTTP/1.1 404 Not Found
Date: Sat, 02 Feb 2019 01:26:57 GMT
Content-Length: 42
Content-Type: text/plain; charset=utf-8

That's not a supported number new Ghoper!

Testing Http endpoints

Testing http endpoints is extremely easy in Go, and doesn’t even require you to actually listen on any ports! The httptest package provides a few handy utilities, including NewRecorder which implements http.ResponseWriter and allows you to effectively make an http request by calling ServeHTTP directly. Here’s an example of a test for our previously implemented numberDumperInt and numberDumperString, commented with what exactly is happening:

Let's test the api above, to see the behavior and how easy it is to use tests using endpoints...

Check the code below:

package main

import (
	"fmt"
	"net/http"
	"net/http/httptest"
	. "testing"
)

func TestNumberDumperInt(t *T) {
	// We first create the http.Handler we wish to test
	n := numberDumperInt(3)

	// We create an http.Request object to test with. The http.Request is
	// totally customizable in every way that a real-life http request is, so
	// even the most intricate behavior can be tested
	r, _ := http.NewRequest("GET", "/one", nil)

	// httptest.Recorder implements the http.ResponseWriter interface, and as
	// such can be passed into ServeHTTP to receive the response. It will act as
	// if all data being given to it is being sent to a real client, when in
	// reality it's being buffered for later observation
	w := httptest.NewRecorder()

	// Pass in our httptest.Recorder and http.Request to our numberDumper. At
	// this point the numberDumper will act just as if it was responding to a
	// real request
	n.ServeHTTP(w, r)

	// httptest.Recorder gives a number of fields and methods which can be used
	// to observe the response made to our request. Here we check the response
	// code
	if w.Code != 200 {
		t.Fatalf("wrong code returned: %d", w.Code)
	}

	// We can also get the full body out of the httptest.Recorder, and check
	// that its contents are what we expect
	body := w.Body.String()

	if body != fmt.Sprintf("DevOps BH, Golang is Life, Here's your number: 3\n") {
		t.Fatalf("wrong body returned: %s", body)
	}
}

In this way it’s easy to create tests for your individual components that you are using to build your application, keeping the tests near to the functionality they’re testing. Note: if you ever do need to spin up a test server in your tests, httptest also provides a way to create a server listening on a random open port for use in tests as well.

Run go

$ go test

Output:

PASS
ok  	net-http/tests-endpoints	0.002s

Http Shutdown Gracefully

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"
	"sync"
	"time"
)

// HTMLServer represents the web service that serves up HTML
type GoServerHttp struct {
	server *http.Server
	wg     sync.WaitGroup
}

func indexHandler(w http.ResponseWriter, req *http.Request) {
	w.Write([]byte(`
	<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Golang/DevOpsBH</title>

  <style>
  body {
  background-color: #424242;
  color: #F6F6F6;
  text-align: center;
  font-family: Helvetica, Arial, sans-serif;
  font-size: 20px;
  }
  h1, h2, h3 {
  margin: 0;
  line-height: 1.5;
  }
  .print-container {
  background-color: rgba(0, 0, 0, .3);
  padding: 15px;
  margin: 30px auto;
  width: 50%;
  border-radius: 4px;
  }
</style>

</head>
<body>
  <div class="print-container">
  <h1>{{ .Name }}</h1>
  <h2>Workshop Golang for DevOps!</h2>
  </div>
</body>
</html>
	`))
}

func Ping(w http.ResponseWriter, req *http.Request) {
	w.Write([]byte(`{"status":"success","msg":"Devops BH for Golang StartServer!"}`))
}

func main() {
	// DefaultServeMux
	mux := http.NewServeMux()

	// POST handler /api/v1/ping
	handlerApiPing := http.HandlerFunc(Ping)

	// handler ping
	mux.Handle("/v1/api/ping", handlerApiPing)

	// templates/index html
	// if you want to activate this handler, the directory templates
	// where the html file is located must
	// be sent next to the binary to work, as it needs to parse the html
	// mux.HandleFunc("/", tpl.ShowHtml)

	// this handler implements the version
	// that does not need the html file
	mux.Handle("/", http.HandlerFunc(indexHandler))

	// Create the HTML Server
	ApiServer := GoServerHttp{
		server: &http.Server{
			Addr:           ":8080",
			Handler:        mux,
			ReadTimeout:    10 * time.Second,
			WriteTimeout:   20 * time.Second,
			MaxHeaderBytes: 1 << 25, //32Mb
		},
	}

	go func() {

		log.Printf("\nServer run :8080\n")
		// service connections
		if err := ApiServer.server.ListenAndServe(); err != nil {
			log.Printf("listen: %s\n", err)
		}
	}()

	var errs = make(chan error, 2)

	go func() {
		// Setting up signal capturing
		c := make(chan os.Signal)
		signal.Notify(c, os.Interrupt)
		errs <- fmt.Errorf("Notify here: %s", <-c)

	}()

	stop := make(chan os.Signal, 1)
	signal.Notify(stop, os.Interrupt)

	// Waiting for SIGINT (pkill -2)
	//<-errs

	// Wait for interrupt signal to gracefully shutdown the server with
	// a timeout of 5 seconds.
	//quit := make(chan os.Signal)
	//signal.Notify(quit, os.Interrupt)
	//<-quit
	<-stop

	log.Println("Shutdown Server ...")
	// ... here is the code to close all
	// ...
	// ....

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	if err := ApiServer.server.Shutdown(ctx); err != nil {
		log.Fatal("Server Shutdown now:", err)
		// ... here is the code to close all error context
		// ...
		// ....
	}

	// execute finish
	log.Println("Server exist")

	<-errs
}

Go to the browser and type:

http://localhost:8080

Making a request in your api:

$ curl -i -Xget localhost:8080/v1/api/ping

HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 02:32:32 GMT
Content-Length: 62
Content-Type: text/plain; charset=utf-8

{"status":"success","msg":"Devops BH for Golang StartServer!"}%

After stopping the program, with CTRL + C, look what will happen.

2019/02/02 00:55:30 
Server run :8080
^C2019/02/02 00:55:31 Shutdown Server ...
2019/02/02 00:55:31 Server exist
2019/02/02 00:55:31 listen: http: Server closed

Now let's try using the kill command

$ ps aux | grep "name-api"
$ kill -SIGINT <PID>

Look at the exit:

2019/02/02 00:52:24 
Server run :8080
2019/02/02 00:55:10 Shutdown Server ...
2019/02/02 00:55:10 listen: http: Server closed
2019/02/02 00:55:10 Server exist

Middleware

Serving endpoints is nice, but often there’s functionality you need to run for every request before the actual endpoint’s handler is run. For example, access logging. A middleware component is one which implements http.Handler, but will actually pass the request off to another http.Handler after doing some set of actions. The http.ServeMux we looked at earlier is actually an example of middleware, since it passes the request off to another http.Handler for actual processing.

There are several ways to implement a Middleware, but the concept behind everything is the same for everyone, we will always have to return a return http.HandlerFunc, all libs have done this way, there are very elegant implementations and several libs on the internet to do this.

Let's implement our Middlewares and see how it works in practice.

Come with me, now that things start to get cool.

Here’s an example of our previous example with some logging middleware:

Check the code below:

package main

import (
	"io"
	"log"
	"net/http"
	"time"
)

// color terminal
var Expre = "\033[5m%s \033[0;103m%s\033[0m \033[0;93m%s\033[0m \033[1;44m%s\033[0m"

func Ping(w http.ResponseWriter, r *http.Request) {

	json := `{"status":"success","msg":"pong"}`
	w.Write([]byte(json))
}

// This middleware is responsible for holding up when we have a
// very large number of accesses in a very small time interval,
// depending on the capacity of your traffic, cpu and memory.
// It is one of the favorite middlewares, it is very powerful,
// not only determines the number of clients, but it does
// not have to lose in the requests sent.
func MaxClients(n int) Adapter {
	sema := make(chan struct{}, n)
	return func(h http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			sema <- struct{}{}
			defer func() { <-sema }()
			h.ServeHTTP(w, r)
		})
	}
}

// This middleware is only a simulation, to implement the
// jwt in Go is very quiet, I will
// demonstrate in other topics below.
func AuthJwt() Adapter {
	//s1 := logg.Start()
	return func(h http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			//if gjwt.CheckJwt(w, r) {
			if r.Header.Get("X-KEY") == "123" {
				h.ServeHTTP(w, r)
			} else {
				msgjson := `{"status":"error","message":"error in Jwt!"}`
				w.Header().Set("Content-Type", "application/json; charset=utf-8")
				w.WriteHeader(http.StatusUnauthorized)
				io.WriteString(w, msgjson)
				//logg.Show(r.URL.Path, strings.ToUpper(r.Method), "error", s1)
			}
		})
	}
}

// Middleware Logger
func Logger(name string) Adapter {
	return func(h http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			start := time.Now()
			h.ServeHTTP(w, r)
			log.Printf(
				"%s %s %s %s",
				r.Method,
				r.RequestURI,
				name,
				time.Since(start),
			)
		})
	}
}

// Middleware Logger
func LoggerColor(name string) Adapter {
	return func(h http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			start := time.Now()
			h.ServeHTTP(w, r)
			log.Printf(
				Expre,
				r.Method,
				r.RequestURI,
				name,
				time.Since(start),
			)
		})
	}
}

type Adapter func(http.Handler) http.Handler

// Middleware
func Middleware(h http.Handler, adapters ...Adapter) http.Handler {
	for _, adapter := range adapters {
		h = adapter(h)
	}
	return h
}

func main() {

	mux := http.NewServeMux()

	handlerApiPing := http.HandlerFunc(Ping)

	// generate token jwt
	// handler token
	mux.Handle("/v1/api/ping",
		Middleware(handlerApiPing,
			Logger("ping"),
		))

	mux.Handle("/v1/api/login",
		Middleware(handlerApiPing,
			AuthJwt(),
			//Logger("login"),
			LoggerColor("login"),
		))

	// show run server
	log.Printf("\nServer run :8080\n")

	// Listen
	log.Fatal(http.ListenAndServe(":8080", mux))
}

$ go run api-server-middleware.go

$ curl -i -Xget localhost:8080/v1/api/login -H "X-KEY: 123"
HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 03:37:12 GMT
Content-Length: 33
Content-Type: text/plain; charset=utf-8

{"status":"success","msg":"pong"}%

$ curl -i -Xget localhost:8080/v1/api/login -H "X-KEY: 123454"
HTTP/1.1 401 Unauthorized
Content-Type: application/json; charset=utf-8
Date: Sat, 02 Feb 2019 03:43:39 GMT
Content-Length: 44

{"status":"error","message":"error in Jwt!"}%

$ curl -i -Xget localhost:8080/v1/api/ping
HTTP/1.1 200 OK
Date: Sat, 02 Feb 2019 03:43:57 GMT
Content-Length: 33
Content-Type: text/plain; charset=utf-8

{"status":"success","msg":"pong"}%

http DetectContentType

DetectContentType implements the algorithm described at mimesniff to determine the Content-Type of the given data. It considers at most the first 512 bytes of data. DetectContentType always returns a valid MIME type: if it cannot determine a more specific one, it returns "application/octet-stream".

func DetectContentType(data []byte) string

Let's now visualize a code that is will simply open the file to discover its content-type.

We see that we can use the http.DetectContentType function to work together even without being an API directly.

Check the code below:

import (
    "fmt"
    "net/http"
    "os"
)

func main() {

    // Open File
    f, err := os.Open("./jeff-super.jpeg")
    if err != nil {
        panic(err)
    }
    defer f.Close()

    // Get the content
    contentType, err := GetFileContentType(f)
    if err != nil {
        panic(err)
    }

    fmt.Println("Content Type: " + contentType)
}

func GetFileContentType(out *os.File) (string, error) {

    // Only the first 512 bytes are used to sniff the content type.
    buffer := make([]byte, 512)

    _, err := out.Read(buffer)
    if err != nil {
        return "", err
    }

    // Use the net/http package's handy DectectContentType function. Always returns a valid
    // content-type by returning "application/octet-stream" if no others seemed to match.
    contentType := http.DetectContentType(buffer)

    return contentType, nil
}

Output:

Content Type: image/jpeg

Name		Name	Last commit message	Last commit date
Latest commit History 127 Commits
examples		examples
img		img
LICENSE		LICENSE
README.md		README.md

License

jeffotoni/goworkshopdevops

Folders and files

Latest commit

History

Repository files navigation

DevOps BootCamp

Golang Workshop DevOps

Telegram:

Slack:

Lab 01 Install and Commands Golang

Lab 02 The Golang Types

Lab 03 Parse with Golang, Yaml, Toml and Json

Lab 04 Building apis with net/http

Lab 05 Using Golang to Create Command Line Programs

Lab 06 Goroutine The Power

Overview

Introduction Installation

Installation

Linux

$GOPATH

Test our Installation

Workspace

Outside $GOPATH

Installation Docker

Install Docker to Golang

Compile Your app Inside The Docker Container

Cross-compile Your app Inside The Docker Container

Example main.go

Introduction Golang

Golang Language

Keywords

Operators and Punctuation

Println Print

Bufio NewWriter

Func Main

Go Commands

Go Commands Introduction

Go Run

Go Build

Go Install

Go Get

Go Mod

Go Mod Init

Go Mod Vendor

GO111MODULE

Go Test

Lab 02 Types with Golang

Types

Numeric Types

String Types

Pointer Types

Array Types

Slice Types

Struct Types

Struct in C

Struct Type Tags Json

Fatih Structs to Map

Map Types

Map Literals Continued

Channel Types

Blank Identifier

Interface Types

Here's an interface as a method

Interface as type

Exercise One

Control Structures

Control

Control Return

Control Goto

Control if Else

Control For Break Continue

Control Switch Case Break

Control Label

Control Range

Errors

Introduction Errors

How Error Control Works

Errors New

Custom Errors

Packages