Security Automation: Best Practices for Secrets Management in a Configuration-as-Code Environment

Software and infrastructure architecture are quickly evolving into cloud environments. With this migration, Infrastructure as Code (IaC) is melding with automated DevOps practices on the application side, supporting fully automated pipelines that configure, build, test, and deploy software systems.

IaC and DevOps automation makes the previously manually managed processes quicker, less prone to human error, highly repeatable, and consistent.

A common requirement of automation is to keep secrets safe and distribute them securely, in a way that can be managed and audited. Fortunately, tools to accomplish this are available now.

In this article, we’ll look at some problems and solutions related to secret management and distribution in automated environments.

Secrets, configuration management, and IaC

Applications, orchestration software, and other tools need secrets to be able to access, build, test and execute applications in your environment

You need to keep these secrets secure, not stored in clear text that gets checked into your source control or deployed in files. Secrets can be in configuration files, scripts, and application source code. They could also be stored in clear text or encrypted in one or more database tables. It is crucial to identify these issues and move secrets to a secure, manageable location.

Tools like Ansible open source have a provision for storing secrets securely. Ansible is a DevOps management tool for provisioning, configuring, and deploying applications. When you have multiple environments, applications, and tools, each may have a secret requirement. One problem is that Ansible secrets storage is not partitioned to give the least privilege to each tool, task, or application. For example, you would not want a test environment to have access to production secrets.

Taking secrets a step further, you would want to manage and audit the access to resources using a clearly defined central policy. Policies help you determine what process or application has read or write access to particular resources.

When dealing with larger systems, managing access at the policy level — while letting tools do the more detailed work of maintaining access policy, passwords, and API keys — makes overseeing a complex system much more straightforward.

Another benefit is an audit trail, which lets you track attempts to gain access contrary to the set policy or retrace access to a resource for forensic review.

CyberArk Conjur supports Role-Based Access Control (RBAC) policies that allow authorization to be as granular and specific as you want.

Secrets management for IaC with CyberArk Conjur

The RedHat Ansible integration with Conjur adds a much more robust capability for managing secrets within Ansible playbooks. Ansible playbooks are YAML formatted text files containing configuration details and execution commands. Using Ansible-specific commands, you can automatically build, test, and deploy applications.

Ansible enables you to securely access secrets that Ansible needs to build and deploy, and also secrets to pass into applications for their operations. These secrets may be for database access, web services, API authentication, or cloud services from AWS or Azure.

Ansible does support the storing of secrets, but it’s not easy to manage many secrets or have visibility into the applications that use them. With CyberArk Conjur, you can limit the Ansible stored secrets to only one that allows access to Conjur. From that point on, secrets can be stored, managed, and obtained from Conjur in a controlled and audited way. For more discussion, description, and information about the Ansible and Conjur integration see the Ansible section of the CyberArk documentation.

Puppet is another automation tool that allows you to manage and automate infrastructure and complex workflows. Using Puppet, you define your systems’ desired state by writing IaC in Puppet’s Domain-Specific Language (DSL). Puppet will run continuously, and if a system configuration drifts, it will apply commands to bring it back to the desired state while reporting what it finds.

Puppet Forge is a catalog of modules created by Puppet and the Puppet user community to allow easy integration and expand Puppet’s capabilities in a DevOps system. A CyberArk Conjur module is available in Puppet Forge to utilize Conjur to secure secrets in your Puppet configurations. The module is installed using Bolt, Code Manager, or the Puppet module tool. More details about this module are found in the article Register Conjur in Puppet.

Terraform is another widely used and popular IaC tool that can be used to configure and provision infrastructure and workflows. To use Terraform, you create configuration files that describe the components and connections. Terraform then runs and creates an execution plan which defines step by step actions to achieve your desired state. Terraform then executes the plan allowing the next steps to proceed when the previous configuration state has been achieved.

There are three cases where using Conjur with Terraform may be useful:

  • If you are using Terraform but don’t have a secrets vault, you can start using Conjur incrementally to mature your security level.
  • If you are currently using Conjur and want to integrate an IaC tool, Terraform is an excellent choice.
  • If you don’t have any secrets vault yet, Conjur is a perfect choice as it integrates with many IaC tools including Terraform.

Find more information about Terraform/Conjur integration in the article Using Conjur with Terraform.

Securing Ansible with Conjur

Let’s take a brief look at the difference between deploying an app using configuration with secrets exposed versus doing the same thing using Conjur to manage secrets for us. In both cases, we will look at some example Ansible playbooks to illustrate the point.

For more information on installing Redhat Ansible see Installing Ansible in the Ansible documentation, and for Conjur see the quick start Installing Conjur Open Source Secrets Management Tutorial.

Here’s a potentially insecure implementation where credentials for a MySQL database in this instance are stored in a YAML file for use in an Ansible playbook. Note there are other aspects of MySQL configuration, such as a machine root password, that are not addressed here.

Under a “db” role directory, some vars might be defined in the vars directory in a file main.yml as such:

db_name: myDB
db_user: myDBUser
db_pass: pw123456

These variables would then be used in a playbook to create a MySQL database and user.

- name: Create database
  mysql_db:
    name: '{{ db_name }}'
  state: present

- name: Create user for the database
  mysql_user:
    name: '{{ db_user }}'
    password: '{{ db_pass }}'
    priv: '{{ db_name }}.*:ALL'
  state: present

This information is stored in a file and likely checked into source control. There are options to pass a pre-encrypted password or store and retrieve with Ansible. Managing these can be a problem if you have multiple instances of MySQL. There will be a widening set of places where passwords are stored on hosts and instances.

A more secure option to keeping secrets in plain-text is to let Conjur manage secrets. Installing and using the Ansible roles for Conjur in an Ansible playbook is simple.

To start, install the Conjur identity role into your Ansible host:

$ ansible-galaxy install cyberark.conjur-host-identity

This command will enable Conjur on the Ansible host. It will also install another CyberArk utility, Summon. Summon is an open-source utility from CyberArk for fetching secrets from trusted stores such as Conjur and other secrets management systems.

When you set up Conjur, it could be on the same machine, but more likely, it is installed somewhere else and accessible via https for the Ansible host node. For example, assume that Conjur is accessed at a URL of https://conjur.yourdomian.com/api.  You have created an account for this Ansible host of “myaccount”, then YAML for making the connection in your Ansible playbook would be:

- hosts: servers
  roles:
    - role: cyberark.conjur-host-identity
      conjur_appliance_url: 'https://conjur.yourdomain.com/api',
      conjur_account: 'myaccount',
      conjur_host_factory_token: "{{lookup('env', 'HFTOKEN')}}",
      conjur_host_name: "{{inventory_hostname}}"

Your Ansible host can now retrieve secrets from Conjur.

With Conjur, secrets are stored in a central repository available to all registered hosts and applications.

To fetch a value in a playbook you can specify conjur_variable and used like this in the playbook:

- name: Create user for the database
  mysql_user:
    name: '{{ db_user }}'
    password: "{{ lookup('conjur_variable', '/path/to/secret') }}"
    priv: '{{ db_name }}.*:ALL'
  state: present

Now secrets are in a central, secure store that can be managed and audited. For more control, you can configure CyberArk Privileged Access Security to help organize, maintain policies, and change passwords.

Next steps

We have seen how CyberArk Conjur can be used to secure secrets in today’s DevOps environment. Conjur can be used on its own, but is more powerful when combined with DevOps and IaC orchestration tools like RedHat Ansible.

We covered some of the steps to use Conjur with Ansible, but we barely scratched the surface of CyberArk Conjur capabilities in the many different environments that require secret management.

To get started with Conjur and Ansible check out our new interactive KataCoda tutorial for Ansible or the “Managing Secrets in Red Hat Ansible Automation Playbooks” tutorial to install it yourself.  There are a lot of great blogs on Conjur.org and interactive tutorials that you should check out and as always, come talk to me in the CyberArk Commons forum.