How to Setup Serverless IAM Authenticator with AWS Lambda

Lambda functions are a great way to build a microservices application without the need to provision or manage servers. You simply create your functions, upload them, and AWS takes care of the rest.

That said, managing secrets in Lambda functions, especially those outside of the normal IAM roles provided by AWS, can be challenging. Fortunately, CyberArk Conjur provides a secure way to centrally store all your secrets while still leveraging simple AWS roles to provide access to them.

To achieve this, Conjur offers a suite of APIs you can call to provide secrets to your Lambda functions when they are initialized, based on the Lambda function’s role. This allows you to manage all your secrets in a central location, making rotation of secrets a lot easier.

In this article, we’ll take a look at an application composed of two main Lambda functions, one of which accesses DynamoDB for data storage, and the other that accesses Simple Email Service (SES) for sending emails.

This article assumes that you are familiar with AWS services, particularly Lambda functions, DynamoDB, and SES.

Setup and Prerequisites

In our previous post, we ran through the setup for a number of components. To follow along with this article, you will need:

DynamoDB with a programmatic user account, as well as some sample data in it (we are using a movie database).
Simple Email Service (SES) with a domain and a sending user.
Conjur installed and running (we are using a simple quick start installation of Conjur using Docker on EC2 as described in the Conjur Quickstart documentation).

With the basic building blocks in place, let’s configure our Conjur instance to support our Lambda function application.

Using the Configure IAM Authenticator

We are going to set up the Conjur instance to manage access to DynamoDB and SES instances, as well as to store the connection properties for both. Before we can do this, we need to set up a number of IAM roles that our functions can use to access the secrets in Conjur. Conjur will use these roles to determine whether the functions are allowed to retrieve the secrets.

We’ve created the following Lambda IAM roles, with no policies attached, to give access to our secrets:

LambdaDynamoDBReadAccess
LambdaSESSendAccess

These roles allow us to provide to our applications access to the different secrets. In a production scenario, you might switch the connection string secrets into separate roles, so as to reuse and rotate them separately if required. In this example, we’ll keep them together.

Next, we need to set up the IAM authenticator in Conjur, which will let our app use its AWS IAM role to authenticate using a process similar to the one documented here.

First, add the following environment variable to enable the connector in your Docker configuration file (where dev is the service ID):

CONJUR_AUTHENTICATORS=authn-iam/dev

We can then create a policy to enable any of our services to use the authenticator by creating the following root-policy.yml file:

# policy id needs to match the convention `conjur/authn-iam/<service ID>`
- !policy
  id: conjur/authn-iam/dev
  body:
  - !webservice
  - !group clients
  - !permit
    role: !group clients
    privilege: [ read, authenticate ]
    resource: !webservice

- !policy
  id: <app_id>
  body:
  - !group iam-authn

- !grant
  role: !group conjur/authn-iam/dev/clients
  member: !group <app_id>/iam-authn

Using the Conjur CLI, load this root policy:

conjur policy load root root-policy.yml

We are also going to create two child policies, one for the DynamoDB instance, and the other for the SES service. For this, we’ll create two policies in the following format:

- &apps
  - !host <AWS_account>/<IAM_role_name>

- &secrets
  - !variable connectionstring
  - !variable username
  - !variable password

- !group secrets

- !permit
  role: !group secrets
  resources: *secrets
  privileges: [ read, execute ]

- !grant
  roles:
  - !group secrets
  - !group iam-authn
  members: *apps

Using this template, we can create two separate application roles for our Lambda functions using the following table to substitute values:

File Name         dynamodb-app.yml              ses-app.yml
<app_id>          dynamoDBApp                   sesApp
<AWS_account>     account #                     account #
<IAM_role_name>   LambdaDynamoDBReadAccess      LambdaSESAccess

Run the following in the Conjur CLI for each of the YAML files:

conjur policy load <app_id> <name>-app.yml

This will add the applications to the Conjur database and set three variables for each application: connectionstring, username, and password.

Before closing the Conjur CLI, populate our instance with the actual secrets by using the following command:

conjur variable values add <app_id>/<variable> <value>

Use this command to populate the values of all six variables — connectionstring, username, and password for each of our two applications.

Creating Authenticated Lambda Functions

Now that Conjur is ready to go, let’s create two simple python Lambda functions that we can deploy to access the Dynamo database and SES service. Conjur has a simple conjur-authn-iam-client-python example in their GitHub repository that we are going to modify to suit our use case.

First, create a basic Lambda Function in Python and follow the steps in installing the Conjur client and building the IAM client using pip as described in the readme file.

pip3 install --user conjur-client
git clone https://github.com/cyberark/conjur-authn-iam-client-python.git
cd conjur-authn-iam-client-python; pip3 install --user .

This will be our database function, so make sure that the IAM role LambdaDynamoDBReadAccess’ is set as the execution role with the relevant Lambda Function permissions.

We also need to set a number of environment variables because Lambda functions are not allowed to call the STS endpoint (as mentioned in the GitHub instructions), so let’s set these as well. Our environment variables look like the following:

CONJUR_APPLIANCE_URL=https://ec2-18-237-182-182.us-west-
2.compute.amazonaws.com:8443
AUTHN_IAM_SERVICE_ID=dev
CONJUR_AUTHN_LOGIN=host/<app_id>/548836857282/LambdaDynomoDBReadAccess
CONJUR_CERT_FILE=./conjur-dev.pem
CONJUR_ACCOUNT=dev
IAM_ROLE_NAME=LambdaDynomoDBReadAccess
IGNORE_SSL=true

As you can see, we also need to import our client self-signed certificate from the Conjur server (or use a proper certificate if you have one). To do that, use the following command (replace the URL shown here with your own):

openssl s_client -showcerts -servername ec2-18-237-182-182.us-west-
2.compute.amazonaws.com -connect ec2-18-237-182-182.us-west-
2.compute.amazonaws.com:8443 < /dev/null 2> /dev/null | sed -ne '/-BEGIN 
CERTIFICATE-/,/-END CERTIFICATE-/p'

In the Lambda method (ours is called lambda_function.py) we can now use the following code to define the handler and initialize the Conjur client:

def lambda_handler(event, context):

    iam_role_name=os.environ['IAM_ROLE_NAME']
    access_key=os.environ['AWS_ACCESS_KEY_ID']
    secret_key=os.environ['AWS_SECRET_ACCESS_KEY']
    token=os.environ['AWS_SESSION_TOKEN']
    conjur_client = create_conjur_iam_client_from_env(iam_role_name, access_key, secret_key, token)

This code defines the various required variables and connects our application to the Conjur client.

The next code block finishes off the function by retrieving the variables we set earlier for the Dynamo DB application, and returns the values:

connection_string = client.get('dev/dynamoDBApp/connectionstring')
username = client.get('dev/dynamoDBApp/username')
password = client.get('dev/dynamoDBApp/password')
return {
  "connectionString": connectionString,
  "username": username,
  "password": password
}

From here, we could use these values to establish a connection to our DynamoDB database. We can also test to see if our DynamoDB function can access the SES function secrets by using the following Try/Catch statement:

try:
    connection_string = client.get('dev/sesApp/connectionstring')
    username = client.get('dev/sesApp/username')
    password = client.get('dev/sesApp/password')
except:
    return { "error": "An exception occurred: Not allowed to retrieve values" }

When the function tries to access those secrets, it will return an error response.

Next Steps

By separating our access to functions away from the AWS roles and using Conjur as a central store, we can reduce the risks associated with having too many AWS roles, as well as simplify secret rotation.

This makes it easier to manage the microservice applications running on AWS Lambda functions securely, without being locked into AWS authorization schemes. It also allows you to centrally manage our secrets across the environments (if you have other resources hosted outside of AWS) while still leveraging AWS identity management.

Conjur provides APIs and secrets management for a wide variety of applications, platforms, and languages outside the scope of this example, but you can find many more tutorials on our blog or try our KataCoda tutorials for a fully interactive learning experience. If you like what you see, signup for our monthly DevOps newsletter to get the latest on new tutorials and blogs or join our developer community forum to continue the conversation.

Andrew Copeland

I am a software engineer who enjoys making developers lives easier and stress free. I believe software should be simple but powerful. I enjoy the outdoors, hiking, fishing and camping.