This article will cover basic steps for creating a time-triggered Lambda function. This can be useful if you want to check for something periodically (for example once per hour). You might want to check if your server is up, fetch data from an API and compare them to previously collected values and so on. The possibilities are endless with a serverless platform like Lambda.
What is AWS Lambda
Let's briefly touch upon what Lambda actually is and how you can access it. Lambda is a serverless platform offered by Amazon Web Services (AWS). By using Lambda, you eliminate the need to provision, set up and administer your own server, you just upload the scripts you want to run to Lambda and the AWS backend takes care of everything related to actually running your script. This is especially useful if you have small scripts which don't run long and provisioning a server just to run them wouldn't make much sense. With Lambda, you pay only for the resources/time said script consumes, as opposed to running it on a dedicated EC2 instance.
As part of the Always Free tier of AWS, you get 1 million requests per month for free on Lambda, which is pretty good. Accessing Lambda is easy, you may do it from the AWS Console or with a command line, for this article I'm going to be using the AWS Console as it is more of a straight-forward approach.
Creating the function
For the sake of simplicity, let's assume we want to create a function which gets values of Bitcoin, Ethereum and Dogecoin from the messari.io API every hour. It would also be nice to save these to a database. Let's also assume we have an instance of MongoDB up and running somewhere. An S3 bucket on AWS or any other database can also be used, you may also just log the values without storing them, that should be enough as a proof of concept.
I will be creating this function in Go, it is also possible to create Lambda functions in Python, .NET Core, Java, Ruby or NodeJS.
The code
First things first, let's get the code done. For creating a function in Go, you'll need the aws-lambda-go package, you can get it by invoking go get github.com/aws/aws-lambda-go/lambda from the command line. The function will probably work even without this library, but even if your script completes successfully, you'll get errors reported by Lambda (trust me, I've tried).
After that, it's just a matter of importing the rest of the libraries needed to make this work and implement the functionality we want. The main function's code would look like this:
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"os"
"time"
"github.com/aws/aws-lambda-go/lambda"
client "github.com/bozd4g/go-http-client"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type Coin struct {
Status struct {
Timestamp time.Time `json:"timestamp"`
} `json:"status"`
Data struct {
Symbol string `json:"symbol"`
Name string `json:"name"`
MarketData struct {
Value float32 `json:"price_usd"`
} `json:"market_data"`
} `json:"data"`
}
func main() {
lambda.Start(handle)
}
func handle() {
assets := [3]string{"btc", "eth", "doge"}
httpClient := client.New("https://data.messari.io/api/v1/")
db, err := mongo.NewClient(
options.Client().ApplyURI(os.Getenv("MONGO_URI")))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = db.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer db.Disconnect(ctx)
for _, asset := range assets {
request, err := httpClient.Get(
"assets/" + asset + "/metrics?fields=symbol,name,market_data")
if err != nil {
panic(err)
}
response, err := httpClient.Do(request)
if err != nil {
panic(err)
}
data := Coin{}
err = json.Unmarshal(response.Get().Body, &data)
if err != nil {
panic(err)
}
collection := db.Database("coins").Collection("coins")
result, err := collection.InsertOne(ctx, data)
if err != nil {
log.Printf("Could not save values for " + data.Data.Symbol)
} else {
log.Printf(
"Saved data for " + data.Data.Symbol +
" as " + fmt.Sprint(result.InsertedID))
}
}
}
If it isn't apparent, I'll shortly explain what the code does. In the handle() function, it initializes the HTTP client for fetching the API data and the MongoDB client for storing it. It then fetches the data from the API, serializes it into the Coin struct which defines our structure of a cryptocurrency (or at least the data we need to fetch/store), then it stores the data in the database and logs the result. The main() function calls the handle() function and does it's Lambda stuff, pretty straightforward.
Note that an environment variable is being used to store the MongoDB connection string, this is important and will come into play when setting the function up in AWS Lambda control panel.
Lambda setup
Now that we are done with the code, let's go to the AWS Console and into Lambda. A new function can be set up here. Let's click Create function, select the Go runtime and let's name it CryptoRateFetcher. Then click Create function again.
You should land at your function's dashboard. Here you can upload your scripts, select triggers and events and configure your function.
First things first, let's build the script, upload it and set the MongoDB URI as a environment variable. You will need to build a Linux binary, as Lambda can only execute these. If you're on Windows, you can use WSL or Docker to create the binary. In the project's directory, we'll need to run:
GOOS=linux GOARCH=amd64 go build -o main .
zip main.zip main
In this example, I'll upload the .zip directly to Lambda, uploading it to S3 and running the function from there is also possible.
The image above shows the function dashboard where you can upload the zip. Note that the Handler is set to hello, this is a default for some reason and you will need to change it to main or however is your main function called. After uploading the code, head over to the Configuration tab and select Environment Variables. There you can add the MONGO_URI variable which will be consumed by the script on execution. You can now save the function by clicking Actions -> Publish new version in the upper right corner of the dashboard. You can now test the function in the Test tab, which should yield a success, if everything has been set up correctly. The output might look like this:
The Trigger
Now that we know our function is working, let's set the timed trigger to start our function every hour. Back on the main function dashboard, hit Add trigger on the left, a dropdown should appear. Choose Event Bridge and create a new Rule. A new form where you can configure your event should appear. Select Schedule expression as your Rule type and configure it accordingly. You can use regular CRON syntax or use rates. After that, click Add and you're pretty much finished.
The last thing to do is to wait for the results it yields. As you can see in the picture below, over the last several hours, we've fetched and stored multiple Bitcoin values, with the time interval being indeed one hour per record.
Wrapping up
In this article, I've demonstrated an easy way to deploy a rather simple script to Amazon Web Services' serverless platform AWS Lambda. I've also touched upon what Lambda actually is and what pros there are to use it with AWS Always Free tier. A way to set up a timed trigger to start the script in a certain interval was shown.
The examples in this article were very simple, indeed, and you can run much more complex scripts or programs in Lambda, there are also options to invoke the function via REST API which were not touched upon in this article, but I may come back to them on a later date. The example script shown in this article will be available in this repo over at my GitHub. If you found this article interesting, you might also want to check out my other Lambda function written in Go which checks if array of user-defined hosts (webservers) are up, in this repo.
Thanks for reading, until next time...
