Rapid AWS Lambda development with Go and Mantil
If you need to develop an AWS Lambda function in Go, take a look at Mantil, a dev kit with staging and database connection included.
My course Master Go is $60 off until Dec 2nd.
Hop over to AppliedGo.com for more info.
AWS Lambda was among the first FaaS services available commercially. Recently, I came across a tool that promises to make Lambda development faster and easier: Mantil. To test the tool and the docs, I created a lambda function that receives a Google Form response and stores the answers in a DynamoDB table.
Project setup
Installation was pleasantly quick and easy via Homebrew. Mantil also has a cloud-side component, but apart from creating an AWS user (if you do not have one already), I did not need to do anything on AWS to install Mantil. Mantil does this by itself through
mantil aws install
The install
command takes an AWS user's API credentials and the AWS region to install the cloud components to. This, and the corresponding uninstall
command, are the only places where the AWS user's API credentials are needed.
Creating a project from scratch
After the cloud-side installation was finished, I created a new project through
mantil new mantil-template-form-to-dynamodb
Inside the new project directory, a sample ping
endpoint waits to be explored. The starting point of a Mantil endpoint is a struct to hold the data to send back and forth. The Ping
struct holds no data – after all, ping only pongs.
Along with the struct, Mantil created also a Default()
method that responds on the root path of the lambda function, /ping
, and more sample methods like a Hello()
method that receives and sends a string.
type Ping struct{}
func New() *Ping {
return &Ping{}
}
/ping
.func (p *Ping) Default() string {
return "pong"
}
/ping/hello
.func (p *Ping) Hello(ctx context.Context, name string) (string, error) {
return "Hello, " + name, nil
}
The base project is ready to run. Calling
mantil deploy
deployed the ping API.
A cool feature of Mantil is the support for staged deployments. When mantil deploy
is invoked for the first time, it asks for a default stage to create. I entered “development”.
After a few seconds, I could then test the /ping
endpoint right on the command line via
mantil invoke ping
and got the result from the Default()
method back. Likewise,
mantil invoke ping/hello/AppliedGo
triggers the Hello()
method.
Creating a new API
Next, I wanted to create a new API that receives a Google Form response and stores the answers in a DynamoDB table.
Mantil generated the code for me via
mantil generate api form
where form
is the name of the new API. This is the generated code:
type Form struct{}
type DefaultRequest struct{}
type DefaultResponse struct{}
func New() *Form {
return &Form{}
}
func (f *Form) Default(ctx context.Context, req *DefaultRequest) (*DefaultResponse, error) {
panic("not implemented")
}
How to send Google Form data to a Lambda function
A Google form was quickly created from one of the available sample forms - the party invitation form.
Small problem: Google forms do not send Webhooks. So I had to create a custom webhook that sends the data to the Lambda function. Luckily, I found a script and the steps to apply it to the Google form here.
I edited the script to call a new endpoint /form/save
with the form data as JSON payload.
var POST_URL = "https://SOME_UINQUE_ID.execute-api.AWS_REGION.amazonaws.com";
function onSubmit(e) {
var form = FormApp.getActiveForm();
var allResponses = form.getResponses();
var latestResponse = allResponses[allResponses.length - 1];
var response = latestResponse.getItemResponses();
var payload = {};
for (var i = 0; i < response.length; i++) {
var question = response[i].getItem().getTitle();
var answer = response[i].getResponse();
payload[question] = answer;
}
console.log(JSON.stringify(payload))
var options = {
"method": "post",
"contentType": "application/json",
"payload": JSON.stringify(payload)
};
UrlFetchApp.fetch(POST_URL + "/form/save", options);
};
What is the required value for POST_URL
? This depends on the active Mantil project stage. The command
mantil stage list
lists all stages along with their endpoints:
| DEFAULT | NAME | NODE | ENDPOINT |
|---------|------|------|-----------------------------------------------------------|
| * | dev | dev | https://blahblah.execute-api.eu-central-1.amazonaws.com |
But at that point, I did not know exactly how the JSON payload looks. So I added a /save
endpoint to the form API that did nothing but logging the received payload.
func (f *Form) Save(ctx context.Context, req *Form) (string, error) {
log.Printf("Save: req is '%+v'", req)
return "", nil
}
This way, I got hold of the raw request data.
{"What is your name?":"Cat R. Pillar","Can you attend?":"Yes, I'll be there","How many of you are attending?":"3","What will you be bringing?":["Drinks"],"Do you have any allergies or dietary restrictions?":"Nope","What is your email address?":"[email protected]"}
Then, using Matt Holt's indispensable JSON-to-Go service to turn the JSON payload into a Go struct. The result looks kind of funny because the Javascript code at Google form side uses the form's questions 1-1 as field names.
type Form struct {
Name string `json:"What is your name?"`
CanYouAttend string `json:"Can you attend?"`
Count string `json:"How many of you are attending?"`
Items []string `json:"What will you be bringing?"`
Restrictions string `json:"Do you have any allergies or dietary restrictions?"`
Email string `json:"What is your email address?"`
}
I could have pimped the JS script to substitute simpler JSON field names for the questions, but… nah. Too lazy.
Don't lose the RSVP's for your party. Or: from struct to DynamoDB
To save the received form data in DynamoDB, I needed a table. Mantil's Go package provides a ready-to-use Key-Value store but I wanted to create and use a table from scratch.
For this, I added some internal fields to the Form struct:
type Form struct {
table *dynamodb.Client
tableName string
tableResourceName *string
}
Note that these are private fields. They do not get in the way when using the Form
struct for receiving the JSON payload.
Then, I created a New()
function that takes care of creating the table.
func New() *Form {
tableName := os.Getenv("TABLE_NAME")
if tableName == "" {
tableName = "MantilPartyTable"
}
table, err := mantil.DynamodbTable(tableName, TableKey, TableSortKey)
if err != nil {
log.Fatalf("Cannot create table: %s", err)
}
return &Form{
table: table,
tableResourceName: aws.String(mantil.Resource(tableName).Name),
}
}
Side note: staged deployments made easy
A really useful feature in this context is that Mantil manages stage-specific environments. In a YAML file, I have specified two different table names for the development and production stages.
project:
stages:
- name: development
env:
TABLE_NAME: MantilPartyDev
- name: production
env:
TABLE_NAME: MantilParty
This way, development and production data get out of each other's way.
Making the Save() function actually save something
Next, I extended the /form/save
endpoint stub from merely logging the data to saving it to DynamoBD. Note the parameter list and return list. I can choose to pass strings or structs as request or response parameters. Mantil does the necessary conversions behind the scenes. Super convenient.
func (f *Form) Save(ctx context.Context, req *Form) (string, error) {
types
is an AWS package for bridging the gap between Go's static typing and DynamoDB's quite dynamic handling of table schemas. items := &types.AttributeValueMemberSS{Value: req.Items}
if items == nil || len(items.Value) == 0 {
items = &types.AttributeValueMemberSS{Value: []string{""}}
}
_, err := f.table.PutItem(context.TODO(), &dynamodb.PutItemInput{
TableName: f.tableResourceName,
Item: map[string]types.AttributeValue{
"name": &types.AttributeValueMemberS{Value: req.Name},
"canattend": &types.AttributeValueMemberS{Value: req.CanYouAttend},
"count": &types.AttributeValueMemberS{Value: req.Count},
"items": items,
"restrictions": &types.AttributeValueMemberS{Value: req.Restrictions},
"email": &types.AttributeValueMemberS{Value: req.Email},
},
})
if err != nil {
log.Printf("Cannot save form: %s", err)
return "Cannot save form", err
}
return fmt.Sprintf("%s saved", req.Name), nil
}
Testing the lambda function from the command line
With all this code in place, I can now run mantil deploy
, and the lambda function is ready to receive form submissions.
Because I did not want to open the AWS console every time I need to check the table, I added a /form/list
endpoint that dumps the table contents as JSON data.
func (f *Form) List(ctx context.Context, req *DefaultRequest) (*[]Form, error) {
Scan
operation, without an optional filter, returns all the records in the table. out, err := f.table.Scan(context.TODO(), &dynamodb.ScanInput{
TableName: f.tableResourceName,
})
if err != nil {
log.Printf("Cannot scan table: %s", err)
}
var forms []Form
for _, item := range out.Items {
forms = append(forms, Form{
Name: (item["name"]).(*types.AttributeValueMemberS).Value,
CanYouAttend: (item["canattend"]).(*types.AttributeValueMemberS).Value,
Count: (item["count"]).(*types.AttributeValueMemberS).Value,
Items: (item["items"]).(*types.AttributeValueMemberSS).Value,
Restrictions: (item["restrictions"]).(*types.AttributeValueMemberS).Value,
Email: (item["email"]).(*types.AttributeValueMemberS).Value,
})
}
return &forms, nil
}
This endpoind allowed me to simply run
mantil invoke form/list
to get the current contents of the table back:
200 OK
[
{
"What is your name?": "Cat R. Pillar",
"Can you attend?": "Yes, I'll be there",
"How many of you are attending?": "3",
"What will you be bringing?": [
"Drinks"
],
"Do you have any allergies or dietary restrictions?": "Nope",
"What is your email address?": "[email protected]"
}
]
I guess it's time to start the party! 🥳
Odds and ends
I tested a few more things, most notably
- a second stage with a different table name, and
- Mantil's built-in testing. Mantil tests follow the standard Go testing convention and can be called with
mantil test
.
I omit the details here as the article is already long enough, don't you think? Have a look at my Mantil-Party-Form repository for more details.
Links
Find the repository with the complete example here:
christophberger/mantil-template-form-to-dynamodb: Receive form data and write it to a DynamoDB table
Mantil links
Homepage: Mantil
Repositories:
Mantil: mantil-io/mantil: Build your AWS Lambda-based Go backends quicker than ever
Mantil Go package: mantil-io/mantil.go: Go SDK for Mantil projects
Happy coding!