secrets management best practices

Secrets Management Best Practices for Machines and Services to Get Secure Access

Published May 20th, 2020 by Theo Despoudis

When we talk about Secrets Management and handling secrets, we ought to think about all the users of the system that have the ability to handle them. “Users” include machines or application services that, quite often, have hardcoded secrets in an environment file; or even worse, inside the code.

That is because machine/services users are the ones who tend to use most of those secrets anyway; we ought to secure them at the highest level. Yet, we often end up placing them in plain sight. This has to change. In this article we are going to explain what Machine identity is, and why it’s important. Then, we are going to list some of the best practices for Secrets Management for Machine and Services. Our goal is to offer a new way to properly handle secrets at scale.

Let’s get started.

What is Machine Identity anyway?

Machine Identity, as the name implies, is a way to identify machines in a scoped and controlled environment. Machines can be anything from an IOT appliance, to a host or an application. Anything that has a chip for a brain (non-human entity) and has access to the network can be assigned to that role.

Machines are programmed to run unattended and they are the primary consumers of secrets and configuration values. It’s really important in that case to carefully evaluate how those secrets are accessed, where they reside, and who controls access. Traditionally, there is a solution called Privileged Access Management (PAM), which is a centralized way to store and manage credentials for human and non-human (machine) access. This solution needs to be suitable for dynamic and scalable cloud environments to properly protect machine and application access. The responsibilities of protecting Machines/Services secrets have increased, both in scale and on demand as the number of applications and devices in deployment increase.

How we can protect our Machine/Services secrets in the best way possible

We can identify a few crucial points when attempting to securely manage secrets for Machine/Services access:

  • Separation of duties: There should be separate credentials for each identity access. Human accounts should be responsible for administering a particular instance or an instance factory. Machine accounts should be the sole consumers of that privilege by having the minimal required permissions to perform a task. For example:

An application named X needs to possess credentials for accessing the database server. Two separate accounts for X should be created. One for the human admin that can change the database credentials on demand, and one for the Machine operator to read them.

  • Role-Based Controls: In addition, all Machine and Human Accounts should be part of a larger permission pool that assigns permissions based on a role. For example, instead of assigning permissions individually on each user or machine, we add them in pools of different types of roles. Each role has a different permission class. Some can read, others can modify, etc. The idea is to simplify policy assignment.
  • Retrieving secrets from storing them in an opaque way: Your application, host or machine at runtime will fetch the secrets and use them to perform other tasks such as communicating with an API or decrypting some data. If those secrets are hard coded in a file, then they can be clearly viewed or leaked. Plus, if we want to alter them, we would need to clone the repository, make the configuration changes, push them for review, deploy them, and restart the application if they don’t reload when those secrets change.

This is a non-trivial secrets management issue that requires the application to know too much about the secrets, and to change every time the secrets change. A better way is to separate this secrets management functionality so that the application is unaware of any secrets required in the codebase.

The application will perform some operations that require secrets to be used, such as a database connection using minimal credentials (only username for example). Then, a broker that runs alongside can intercept the connection and provide a secret to complete the connection string to the target database. In that way we are relieving our application from managing secrets, once and for all.

  • Adding more than one authentication factor: Just as humans need multi-factor authentication methods to prove who they are (e.g., something they have, know, are), then machines need to have multiple ways to identify their existence. This is because we want to eliminate machines disguised as humans – i.e., bots – that imitate an account.
  • Encryption of secrets at rest and in transit: Proper encryption mechanisms should be in place for transparent encryption of sensitive data. That means that a valid and modern certificate suite to encrypt data should be used. The certificates themselves should be stored, and in the secret store with proper permissions. Ensure that you have a mechanism to recover lost or deleted objects as a safeguard. Also ensure that all communication paths and endpoints between machines and the key vault are secured.
  • Compliance reporting and preventive controls: There should be checks for auditing reasons, whenever a permission or a rule has been changed. This is to showcase the complete trail of actions that lead to the current state, including who, when, why, and what. There should be an easy way to revoke credentials for machines that misbehave or are under attack.

One obvious question that arises from those points is: Is there already a tool or a service that offers those best practices out-of-the-box, and is suitable for enterprises? Yes, there is. CyberArk Conjur is the most enterprise-ready option for secure delivery of secrets in-the-cloud, in-house or hybrid infrastructures. Together, with tools like the Secretless Broker, they manage to remove the friction of secrets management and data theft risks.

Final thoughts

We can understand now, that all the operations that handle secrets in the enterprise for machine/services are very sensitive in nature. They need to be carefully assessed and managed, using the best-known practices. Hopefully, this article helped paint a clear picture of what the current best practices are for managing secrets at that level. By leveraging tools like CyberArk Conjur, which is the defacto solution for application access management of secrets, we can rest assured that those operations run within optimal isolation levels and security.