When Elasticsearch security is enabled for a cluster that is running with a production license, the use of TLS/SSL for transport communications is obligatory and must be correctly setup. Additionally, once security has been enabled, all communications to an Elasticsearch cluster must be authenticated, including communications from Kibana and/or application servers.
The simplest way that Kibana and/or application servers can authenticate to an Elasticsearch cluster is by embedding a username and password in their configuration files or source code. However, in many organizations, it is forbidden to store usernames and passwords in such locations. In this case, one alternative is to use Public Key Infrastructure (PKI) (client certificates) for authenticating to an Elasticsearch cluster.
Configuring security along with TLS/SSL and PKI can seem daunting at first, and so this blog gives step-by-step instructions on how to: enable security; configure TLS/SSL; set passwords for built-in users; use PKI for authentication; and finally, how to authenticate Kibana to an Elasticsearch cluster using PKI.
Enabling securityIn order to enable security it is necessary to have either a Gold or Platinum subscription , or a trial license enabled viaKibana orAPI. For example, the following command would enable a trial license via the API:
curl -X POST "localhost:9200/_xpack/license/start_trial?acknowledge=true"Where localhost must be replaced with the name of a node in our Elasticsearch cluster.
After enabling a license, security can be enabled. We must modify the elasticsearch.yml file on each node in the cluster with the following line:
xpack.security.enabled: trueFor a cluster that is running inproduction mode with a production license, once security is enabled, transport TLS/SSL must also be enabled. On the other hand, if we are running with a trial license, then transport TLS/SSL is not obligatory.
If we are running with a production license and we attempt to start the cluster with security enabled before we have enabled transport TLS/SSL, we will see the following error message:
Transport SSL must be enabled for setups with production licenses. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]Configuration of TLS/SSL is covered in the following sections.
TLS/SSL encryptionElasticsearch has two levels of communications, transport communications and http communications. The transport protocol is used for internal communications between Elasticsearch nodes, and the http protocol is used for communications from clients to the Elasticsearch cluster. Securing these communications will be discussed in the following paragraphs.
Transport TLS/SSL encryptionThe transport protocol is used for communication between nodes within an Elasticsearch cluster. Because each node in an Elasticsearch cluster is both a client and a server to other nodes in the cluster, all transport certificates must be both client and server certificates. If TLS/SSL certificates do not have Extended Key Usage defined, then they are already defacto client and server certificates. If transport certificates do have an Extended Key Usage section, which is often the case for CA-signed certificates used in corporate environments, then they must explicitly enable both clientAuth and serverAuth .
Elasticsearch comes with a utility called elasticsearch-certutil that can be used for generating self-signed certificates that can be used for encrypting internal communications within an Elasticsearch cluster.
The following commands can be used for generating certificates that can be used for transport communications, as described in this page on Encrypting Communications in Elasticsearch :
bin/elasticsearch-certutil ca ENTER ENTER bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 ENTER ENTER ENTEROnce the above commands have been executed, we will have TLS/ SSL certificates that can be used for encrypting communications.
The newly created certificates should be copied into a sub-directory called certs located within the config directory. The certificates will then be specified in the elasticsearch.yml file as follows:
xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12Now restart all of the nodes in our Elasticsearch cluster for the above changes to take effect.
Define built-in user’s passwordsWe must now define passwords for the built-in users as described in Setting built-in user passwords . If we are running with a Gold or Platinum license, the previous steps to enable TLS/SSL for the transport communications must be executed before the cluster will start. Additionally, defining built-in user’s passwords should be completed before we enable TLS/SSL for http communications, as the command to set passwords will communicate with the cluster via unsecured http.
Built-in users passwords can be setup with the following command:
bin/elasticsearch-setup-passwords interactiveBe sure to remember the passwords that we have assigned for each of the built-in users. We will make use of the elastic superuser to help configure PKI authentication later in this blog.
Http TLS/SSL encryptionFor http communications, the Elasticsearch nodes will only act as servers and therefore can use Server certificates ― i.e. http TLS/SSL certificates do not need to enable Client authentication.
In many cases, certificates for http communications would be signed by a corporate CA. It is worth noting that the certificates used for encrypting http communications can be totally independent from the certificates that are used for transport communications.
To reduce the number of steps in this blog, we’ll use the same certificates for http communications as we have already used for the transport communications. These are specified in the elasticsearch.yml file as follows:
xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.client_authentication: optional Enabling PKI authenticationAs discussed in Configuring a PKI Realm , the following must be added to the elasticsearch.yml file to allow PKI authentication.
xpack.security.authc.realms.pki1.type: pki Combined changes to elasticsearch.yml Once the above steps have been followed, we should have the
