Tutorial - Cloud Foundry Integration

Introduction

Cloud Foundry (CF) is a popular open source Platform as a Service (PaaS) that can be deployed on your own infrastructure or on any Infrastructure-as-a-Service platform, like Amazon Web Service, Google Cloud Platform, or Azure. Using Cloud Foundry makes it easy to run, scale, and maintain applications - and integrating it with Conjur makes it easy for CF-deployed apps to securely get the secrets and credentials that they need.

There are several certified CF platforms offering proprietary implementations of Cloud Foundry that simplify the process of installing and maintaining the platform. For the purposes of this tutorial, we will focus on using the small footprint version of Pivotal Cloud Foundry (PCF) called PCF Dev.

In Cloud Foundry, applications interact with external services via service brokers that are registered in the marketplace. Deployments of applications in CF are split up into orgs (the top-level organizational structure where apps can be grouped) and spaces (where users’ applications are actually deployed; spaces belong to orgs). When you deploy an application to a space in CF, if you want to use an external service you must first create a service instance in the space you are working in. Then, when you push (or upload) your application to CF, you can bind your application to the service instance. For more information about how services work in Cloud Foundry, please reference the CF documentation.

Conjur leverages this framework to allow CF-deployed applications to securely access secrets stored in Conjur. We offer the Conjur Service Broker, which can be installed in your CF deployment and used to create Conjur service instances that you can bind your application to. When you bind your application to the Conjur service instance, your application will automatically be granted a Host identity in Conjur policy, and you can add entitlements to the application Host in policy to grant it access to secrets stored in Conjur.

We also offer the Conjur Buildpack, which can be used to automatically inject secret values into your application’s environment at runtime using Summon. If your application is set up to retrieve secret values from its environment, then you can update your app to include a secrets.yml file that provides a map from environment variable name to the secret path in Conjur. The Conjur Buildpack will then automatically use the application’s Host identity to securely retrieve the secret values from Conjur and inject them into the application’s environment at runtime. In practice, as an alternative to using the Buildpack we also have client libraries in several languages that may be used to deliver secrets to your application, but for the purposes of this tutorial we will use the Conjur Buildpack.

Now that we have a better understanding of how our Cloud Foundry integration works, let’s jump right into the tutorial and try it out.

Prerequisites

Before getting started with the tutorial, you must:

The following steps will set up our CF deployment to allow a demo CF-deployed application to interact with Conjur:

A Conjur Admin User will:

  • Configure the Conjur instance to have a Host identity for the Service Broker that owns a Policy; the Service Broker will use this Host to manage application hosts on bind / unbind.

A CF Admin User will:

  • Deploy the Conjur Service Broker to its own org and space and configure it with the Conjur Service Broker host credentials
  • Upload the meta-buildpack and the Conjur Buildpack
  • Create a Conjur service listing in the marketplace
  • Ensure the CF developer user has access to an org / space to deploy their application

A CF Developer User will:

  • Create a Conjur service instance in the org / space where the app will be deployed
  • Deploy the demo application and bind it to the Conjur service instance
  • Request an update to Conjur Policy to add appropriate entitlements to the app’s Host identity in Conjur, and get those changes approved by a Conjur admin before starting or restaging the application

Configure Conjur

We will begin by preparing our Conjur instance by loading a policy file that contains the configuration we will need for our CF-deployed apps. The policy file will include:

  • A cf Policy where all Conjur policies for CF-deployed apps may be loaded. As apps are bound to a service instance of the Conjur Service Broker, they will be granted machine identities and added as Hosts to this Policy.
  • A cf-service-broker Host that is a member of a cf-admin-group Group. The cf-admin-group owns the cf Policy, so it has full privileges on that Policy. The Service Broker will be configured with the credentials of the cf-service-broker Host so that it will be able to modify the cf Policy to manage the application Hosts.

Save the following file as policy.yml:

- !host
  id: cf-service-broker
  annotations:
    platform: cloudfoundry

- !group
  id: cf-admin-group

- !grant
  role: !group cf-admin-group
  member: !host cf-service-broker

- !policy
  id: cf
  owner: !group cf-admin-group

Load the policy as an admin Conjur user using the following command:

$ conjur policy load root policy.yml
Loaded policy 'root'
{
  "created_roles": {
    "default:host:cf-service-broker": {
      "id": "default:host:cf-service-broker",
      "api_key": "2k60y0338tad17178xb9726pjf00jb7dzk2dgkxs43nsmb7e3txmsjc"
    }
  },
  "version": 1
}

Note: For more information about using the Conjur CLI to load policies, work through our getting started tutorial.

We can see from this response that a cf-service-broker Host has been created with API key 2k60y0338tad17178xb9726pjf00jb7dzk2dgkxs43nsmb7e3txmsjc - we will need this key when configuring our Service Broker.

In addition, the demo application that we will use has the following Conjur policy:

- !policy
  id: app
  body:
  - &variables
    - !variable
      id: database/username
    - !variable
      id: database/password
    - !variable
      id: stripe/private_key

  - !group secrets-users
  - !group secrets-managers

  # secrets-managers has role secrets-users
  - !grant
    role: !group secrets-users
    member: !group secrets-managers

  # secrets-users can read and execute
  - !permit
    resource: *variables
    privileges: [ read, execute ]
    role: !group secrets-users

  # secrets-managers can update (and read and execute, via role grant)
  - !permit
    resource: *variables
    privileges: [ update ]
    role: !group secrets-managers

The app requires access to three secrets, and we declare a secrets-users group that can access the secret values, and a secrets-managers group that can update the secret values.

Once our application has a Host identity in Conjur, we will add the Host to the secrets-users group so that it will be privileged to access the secret values.

If this policy is stored in a file app.yml, it can be loaded into the cf Policy by calling

$ conjur policy load cf app.yml
Loaded policy 'cf'
{
  "created_roles": {
  },
  "version": 1
}

The secret values can be populated by calling

$ conjur variable values add cf/app/database/username 'a database username'
Value added
$ conjur variable values add cf/app/database/password 'a database password'
Value added
$ conjur variable values add cf/app/stripe/private_key 'a stripe private_key'
Value added

Configure Conjur Service in Cloud Foundry

Next, we will need to start PCF Dev and log in via the cf CLI.

Note: If you already have PCF Dev running and want to start with a clean installation, you can run cf dev destroy before calling the commands below.

$ cf dev start
Using existing image.
Allocating 4096 MB out of 16384 MB total system memory (6755 MB free).
Importing VM...
Starting VM...
Provisioning VM...
Waiting for services to start...
7 out of 58 running
7 out of 58 running
7 out of 58 running
7 out of 58 running
40 out of 58 running
56 out of 58 running
58 out of 58 running
 _______  _______  _______    ______   _______  __   __
|       ||       ||       |  |      | |       ||  | |  |
|    _  ||       ||    ___|  |  _    ||    ___||  |_|  |
|   |_| ||       ||   |___   | | |   ||   |___ |       |
|    ___||      _||    ___|  | |_|   ||    ___||       |
|   |    |     |_ |   |      |       ||   |___  |     |
|___|    |_______||___|      |______| |_______|  |___|
is now running.
To begin using PCF Dev, please run:
   cf login -a https://api.local.pcfdev.io --skip-ssl-validation
Apps Manager URL: https://apps.local.pcfdev.io
Admin user => Email: admin / Password: admin
Regular user => Email: user / Password: pass

$ cf login -a https://api.local.pcfdev.io --skip-ssl-validation
API endpoint: https://api.local.pcfdev.io

Email> admin

Password>
Authenticating...
OK

Select an org (or press enter to skip):
1. pcfdev-org
2. system

Org> 1
Targeted org pcfdev-org

Targeted space pcfdev-space



API endpoint:   https://api.local.pcfdev.io (API version: 2.82.0)
User:           admin
Org:            pcfdev-org
Space:          pcfdev-space

Step 1: Deploy the Conjur Service Broker to its own org and space

While logged in to PCF Dev as the admin user, run the following commands to deply the Conjur Service Broker:

# Create the org and space in CF for the Service Broker
cf create-org cyberark-conjur-org
cf target -o cyberark-conjur-org
cf create-space cyberark-conjur-space
cf target -o cyberark-conjur-org -s cyberark-conjur-space

# Get the latest release of the Conjur Service Broker and upload to CF
curl -L $(curl -s https://api.github.com/repos/cyberark/conjur-service-broker/releases/latest | grep zipball_url | awk '{print $NF}' | sed 's/",*//g') > conjur-service-broker.zip
unzip conjur-service-broker.zip
repo_dir=$(ls | grep cyberark-conjur-service-broker)
mv $repo_dir conjur-service-broker

# Push the Service Broker app
pushd conjur-service-broker
  cf push --no-start --random-route
popd
rm -rf conjur-service-broker*

# Configure the Service Broker
# This assumes you are using the hosted eval Conjur; if you are running your
# own local Conjur instance at localhost:8080 following the Install Conjur
# tutorial, the URL will be "http://host.pcfdev.io:8080" and the ACCOUNT will
# be "quick-start"
cf set-env conjur-service-broker SECURITY_USER_NAME TEMP_USER
cf set-env conjur-service-broker SECURITY_USER_PASSWORD TEMP_PASS
cf set-env conjur-service-broker CONJUR_ACCOUNT "myname@email.com"
cf set-env conjur-service-broker CONJUR_APPLIANCE_URL "https://eval.conjur.org"
cf set-env conjur-service-broker CONJUR_AUTHN_LOGIN host/cf-service-broker
cf set-env conjur-service-broker CONJUR_AUTHN_API_KEY "2k60y0338tad17178xb9726pjf00jb7dzk2dgkxs43nsmb7e3txmsjc"
cf set-env conjur-service-broker CONJUR_VERSION 5
cf set-env conjur-service-broker CONJUR_POLICY cf
cf set-env conjur-service-broker CONJUR_SSL_CERTIFICATE ""

# Start the Service Broker app
cf start conjur-service-broker

# Create the Service Broker
# Note that the basic auth credentials passed to the `cf create-service-broker`
# command here should match how the Service Broker was configured above
APP_URL="http://`cf app conjur-service-broker | grep -E -w 'urls:|routes:' | awk '{print $2}'`"
cf create-service-broker conjur-service-broker "TEMP_USER" "TEMP_PASS" $APP_URL

Note that in order to be configured properly, the Conjur Service Broker requires several environment variables. These are:

  • SECURITY_USER_NAME / SECURITY_USER_PASSWORD: the Basic Auth credentials for the Service Broker
  • CONJUR_ACCOUNT, CONJUR_APPLIANCE_URL, CONJUR_AUTHN_LOGIN, CONJUR_AUTHN_API_KEY: the credentials needed to access the Conjur instance
  • CONJUR_VERSION: the major version of the instance of Conjur that you are running, which is 5 for this tutorial
  • CONJUR_POLICY: the Conjur Policy where Hosts will be added
  • CONJUR_SSL_CERTIFICATE: not needed for this demo, but can be used if your Conjur instance is secured with TLS. This environment variable should be a copy / paste of the contents of the PEM certificate file that was created on calling conjur init.

Step 2: Upload the meta-buildpack and the Conjur Buildpack

By default, if your application has a secrets.yml file the Conjur Service will use the Conjur Buildpack to automatically inject secret’s into your application’s environment at runtime. The Conjur Buildpack is a decorator buildpack which relies on the meta-buildpack to be properly invoked.

To upload the buildpacks:

# Get the meta-buildpack and upload to cf
curl -L $(curl -s https://api.github.com/repos/cf-platform-eng/meta-buildpack/releases/latest | grep zipball_url | awk '{print $NF}' | sed 's/",*//g') > meta-buildpack.zip
unzip meta-buildpack.zip
repo_dir=$(ls | grep cf-platform-eng-meta-buildpack)
mv $repo_dir meta-buildpack

pushd meta-buildpack
  ./build
  ./upload
popd
rm -rf meta-buildpack*

# get the Conjur buildpack and upload to cf
mkdir conjur-buildpack
pushd conjur-buildpack
  curl -L $(curl -s https://api.github.com/repos/cyberark/cloudfoundry-conjur-buildpack/releases/latest | grep browser_download_url | grep zip | awk '{print $NF}' | sed 's/",*//g') > conjur-buildpack.zip
  unzip conjur-buildpack.zip
  ./upload.sh
popd
rm -rf conjur-buildpack

Step 3: Create a Conjur service listing in the marketplace

To make the Conjur service listing available in all orgs and spaces, run

$ cf enable-service-access cyberark-conjur

Then you will be able to see the cyberark-conjur service listed from any org / space, so that developers will be able to create Conjur service instances:

$ cf marketplace
Getting services from marketplace in org cyberark-conjur-org / space cyberark-conjur-space as admin...
OK

service             plans                  description
cyberark-conjur     community              An open source security service that provides secrets management, machine-identity based authorization, and more.

TIP:  Use 'cf marketplace -s SERVICE' to view descriptions of individual plans of a given service.

Step 4: Create an org / space for the CF developer user

# Create the demo org / spaces
$ cf create-org demo-org
$ cf target -o demo-org
$ cf create-space demo-space
$ cf target -o demo-org -s demo-space

# Give the user a developer role in the demo org / space
$ cf set-space-role user demo-org demo-space SpaceDeveloper

Uploading the Demo Application to Cloud Foundry

The demo application that we are using is simple - it just echoes three environment variables to the browser window. It will allow us to demonstrate that the Conjur service successfully loads the secret values from Conjur into the application environment.

Step 1: Create a Conjur service instance in the org / space for the demo app

To deploy the application, we will act as a developer and log in to CF using the non-admin credentials (user and pass for PCF Dev)

$ cf login -a https://api.local.pcfdev.io --skip-ssl-validation
API endpoint: https://api.local.pcfdev.io

Email> user

Password>
Authenticating...
OK

Targeted org demo-org

Targeted space demo-space



API endpoint:   https://api.local.pcfdev.io (API version: 2.82.0)
User:           user
Org:            demo-org
Space:          demo-space

# Verify that the Conjur service listing is available
$ cf marketplace
Getting services from marketplace in org cyberark-conjur-org / space cyberark-conjur-space as admin...
OK

service             plans                  description
cyberark-conjur     community              An open source security service that provides secrets management, machine-identity based authorization, and more.

TIP:  Use 'cf marketplace -s SERVICE' to view descriptions of individual plans of a given service.

# Create the Conjur service instance
$ cf create-service cyberark-conjur community conjur

Step 2: Deploy the demo application and bind it to the Conjur service instance

The demo application we will use is part of our demo repository, which includes demos for testing our Cloud Foundry integration locally using PCF Dev or against an existing PCF installation.

The app includes a Cloud Foundry deployment manifest that binds it to the Conjur service:

---
applications:
- name: hello-world
  services:
  - conjur

As soon as we cf push the app, it will be bound to the Conjur service instance we created in the last step and it will be granted a host identity in Conjur.

# Download the demo app
curl -L https://github.com/cyberark/cloudfoundry-conjur-demo/archive/master.zip > conjur-demo.zip
unzip conjur-demo.zip
pushd cloudfoundry-conjur-demo-master/app
  cf push --no-start
popd
rm -rf cloudfoundry-conjur-demo-master conjur-demo.zip

Note: We use cf push --no-start here to ensure the app will not start until we have added entitlements for the application Host in Conjur policy so that it will have access to the secrets that it needs.

Step 3: Update Conjur policy to privilege the app

Now that the demo app has a Host identity in Conjur, we can add it as a member of the cf/app/secrets-users group so that it will have access to the secrets in the cf/app Policy. The host identity is stored in the authn_login field in the cyberark-conjur credentials in the application’s environment, and might look something like host/cf/0299a19d-7de4-4e98-89f6-372ac7c0521f. We can update privileges for the app by creating an entitlements.yml policy file:

# Application layer has the secrets-users role
- !grant
  role: !group app/secrets-users
  member: !host 0299a19d-7de4-4e98-89f6-372ac7c0521f

and a Conjur user with privileges to update the cf Policy can load it by calling

$ conjur policy load cf entitlements.yml
Loaded policy 'cf'
{
  "created_roles": {
  },
  "version": 3
}

Step 4: Start the demo app and watch it retrieve secrets!

Start the demo app by calling cf start hello-world. The application will start and will output a URL (like hello-world.local.pcfdev.io). If everything worked correctly, you will see the variable values we loaded into Conjur earlier in this exercise echoed to the browser as planned. For fun, you can rotate the values in Conjur and call cf restage hello-world to watch the values change.

Summary

Developers love Cloud Foundry for the ease of deploying applications, and the Conjur integration enables them to do so securely. Once the Conjur Service Broker and Conjur Buildpack have been installed in Cloud Foundry by an admin user and the Conjur service listing has been made available in the marketplace, it is easy for developers to bind their applications to a Conjur service instance. This gives their application a unique Host identity in Conjur, and that Host identity can be granted access to the secrets that the application needs. At runtime the Conjur Buildpack will install Summon so that the secret values will automatically be loaded into the running application’s environment. With Cloud Foundry and Conjur together, developers can focus on releasing features, and security and operations teams will know that credentials are secure.