Fort Knox for Your Data: How Elasticsearch X-Pack Locks Down Your Cluster – Part 1

Picture this: an always-awake, never-tired, high-speed librarian that instantly finds the exact information you need from a massive collection of books. This extraordinary librarian is also capable of processing millions of requests simultaneously, understands partial or misspelled words, and even predicts what you’re looking for before you finish asking.

Now, imagine that instead of books, this librarian is sorting through enormous amounts of data such as customer records, transaction logs, security events, social media posts, and much more. This is exactly what Elasticsearch does. It is a highly efficient, distributed search and analytics engine designed to rapidly index, search, and analyze vast datasets in real-time. Elasticsearch is built for speed and flexibility, making it indispensable for businesses handling massive data streams that traditional databases struggle with, especially when it comes to complex queries at scale.

However, as Elasticsearch deployments grow, so do the security risks associated with unauthorized access, data breaches, and insider threats.

Suppose Elasticsearch is the tireless librarian retrieving information at lightning speed, then X-Pack is the buff, imposing security officer standing guard at the library’s entrance (as if it were Fort Knox) to ensure that the librarian remains unscathed from threats to perform their duties to the highest possible level.

This powerful extension enforces authentication, authorization, encryption, auditing, and many more (even non-security features such as machine learning). Just as a vigilant security guard prevents unauthorized visitors from wandering into restricted sections, X-Pack ensures that sensitive data remains protected to shield Elasticsearch implementations from potential cyber threats.

Elasticsearch Overview

Elasticsearch has somewhat become a pillar of modern data-driven organizations that power search, analytics, and real-time monitoring across industries. Its scalability allows businesses to handle petabytes of data across multiple nodes, ensuring high availability and fault tolerance.

The powerful full-text search capability supports fuzzy matching, auto-complete, and synonyms, making searches more intuitive and effective. In addition, Elasticsearch provides real-time data processing, which allows businesses to track trends, detect anomalies, and gain insights instantly.

Furthermore, Elasticsearch has built-in aggregations that can perform complex analytics, such as statistical computations and data, visualizations without needing a separate database.

Moreover, the schema-less design of its underlying NoSQL architecture makes it highly flexible, allowing organizations to adapt quickly to changing data structures. Essentially, Elasticsearch has become a crucial tool for data-driven decision-making, whether it’s helping social media platforms deliver personalized content, enabling cybersecurity teams to detect threats, or allowing financial firms to analyze market trends in real time.

X-Pack Plugin Overview

X-Pack is an essential extension that enhances the capabilities of Elasticsearch by providing a suite of advanced features designed to improve security, monitoring, alerting, reporting, machine learning, and more.

In addition, X-Pack seamlessly integrates with the Elastic Stack, which extends its functionality beyond search and analytics to ensure high availability, improved performance, and proactive security enforcement.

X-Pack’s Security Features

X-Pack has comprehensive security features that act as a first line of defense against unauthorized access and data breaches. For example, the authentication and authorization security features restrict access to sensitive data, which ensures that users only interact with the information they are permitted to see.

Expanding on this, "Field and Document Level Security" allows organizations to enforce fine-grained access control by restricting specific fields or entire documents within an index based on a user’s role.

This ensures that even if a user has access to an index, they can only view the portions of data they are authorized to see, which is an essential feature for businesses handling confidential or classified information. In addition, the audit logging feature keeps track of every action within the system, allowing administrators to monitor for suspicious activity and maintain compliance with regulatory standards.

Furthermore, X-Pack’s IP filtering further secures Elasticsearch clusters by blocking unauthorized connections and ensuring communications between nodes are deliberately configured. For organizations operating in highly regulated industries, X-Pack also supports FIPS 140-2 compliance, which is a U.S. government standard for cryptographic security.

This ensures that X-Pack's encryption modules meet rigorous security requirements that make Elasticsearch deployments more compliant with industry regulations that adopt FIPS 140-2-like policies such as the Health Insurance Portability and Accountability Act), Federal Risk and Authorization Management Program (FedRAMP), and General Data Protection Regulation (GDPR).

These security measures make X-Pack an indispensable tool for organizations that depend on Elasticsearch while needing enterprise-grade protection to safeguard their data from cyber threats.

Docker Setup

Cybersecurity teams use Docker to test vulnerabilities by creating isolated, reproducible environments. It allows security professionals to simulate attacks, run exploits, and analyze responses without affecting the host system. Docker's containerization ensures consistent testing conditions, making identifying, assessing, and mitigating security flaws easier.

Teams can use Docker to carry out experiments, allowing cybersecurity professionals to have first-hand experience conducting simulated attacks when protection mechanisms such as X-Pack are disabled or observe the defensive measures implemented when X-Pack is enabled. The goal of this simulation is to create protected and unprotected Elasticsearch environments to demonstrate their safeguards and weaknesses, respectively.

For this simulation, the Docker setup starts with configuring the `elasticsearch.yml` file, which defines the essential settings for the Elasticsearch instance. As shown in Figure 1, the ` elasticsearch.yml` configuration file sets ` xpack.security.enabled` to `false,` which explicitly disables X-Pack security.

Therefore, authentication is not enforced, which leaves the instance open to unauthenticated access. Furthermore, the ` cluster.name` is set to `docker-cluster` to assign a custom name to the Elasticsearch cluster.

Lastly, ` network.host` is set to `0.0.0.0` to configure the instance to listen on all available network interfaces to make it accessible from external sources.

Figure 1. An `elasticsearch.yml` file with X-Pack disabled.

As shown in Figure 2, the ` initialization.sh` script was used to automate the setup of the Elasticsearch target index and sample data.

First, the script starts Elasticsearch in the background and waits for it to become accessible. Once Elasticsearch is ready for access, the script creates a new index named ` secrets` with a single shard and no replicas.

It then populates the ` secrets` index with three sample records, each containing a username and a password. Finally, the script keeps the container running using ` tail -f /dev/null` to ensure that the process does not exit.

Figure 2. The `initialization.sh` file.

Finally, with the complete setup of the `elasticsearch.yml` file and `initialization.sh` file, a `docker run` command can now be executed.

As shown in Figure 3, the ` docker run` command launches an Elasticsearch container with all the required configurations. It runs the container in detached mode ( -d ) and is named ` elasticsearch-xpack.`

The command maps ports 9200 and 9300, which are the default Elasticsearch ports. Furthermore, the ` -e` flags set environment variables, including ` discovery.type=single-node` for standalone operations.

The configuration files are mounted using the ` -v` flag to ensure that the ` elasticsearch.yml` and ` initialize.sh` files are available for the container. In addition, the command also specifies the Elasticsearch image ( 8.17.0 ) from the official Elastic repository. Finally, it runs a shell command ( /bin/bash -c "./initialize.sh && elasticsearch" ) to execute the ` initialize.sh` script before starting Elasticsearch.

Figure 3. The `docker run` command execution.

Accessing the Elasticsearch RESTful API

Now that we have a running Elasticsearch Docker instance, the Elasticsearch RESTful API could be used as a litmus test to determine if everything is running as expected.

As shown in Figure 4, the simple `curl` command responds with a variety of data regarding the running Elasticsearch target. Interacting with Elasticsearch through the RESTful API using `curl` is one of the most fundamental and powerful ways to manage, query, and administer an Elasticsearch cluster.

Furthermore, the Elasticsearch API allows administrators to modify cluster-wide settings dynamically without requiring a restart. This is particularly useful for this article’s use cases as well as multiple implementations of temporary configurations to demonstrate various attack and defense simulations from disabling and enabling X-Pack, respectively.

Another useful function of the Elasticsearch API is confirming configuration, such as validating that this Elasticsearch target has X-Pack disabled, as shown in Figure 5.

Figure 4. A basic Elasticsearch API query test.

Figure 5. Confirmation of the disabled X-Pack security configuration.

Unauthenticated Attack Vectors (X-Pack Disabled)

We can simulate a plethora of attack vectors with a disabled Elasticsearch X-Pack configuration.

Enumerated below are the unauthenticated attack simulations that we will conduct in this article. This list was randomly created and does not comply with any metric, such as severity or difficulty.

1. Accessing Cluster Information
2. Retrieval of All Indices
3. Reading Data from an Index
4. Modifying Index Data
5. Executing Arbitrary Queries
6. Dumping Configuration
7. Exploiting Script Injection
8. Deleting an Index

Accessing Cluster Information

The ability of an attacker to access Elasticsearch cluster information poses significant security risks. As shown in Figure 6, the `curl -X GET "http://localhost:9200/_cluster/health" ` command reveals critical details about the cluster's status, including node availability and active shards. Such insights enable attackers to identify weaknesses, such as unassigned shards or degraded nodes, which can be exploited to launch targeted attacks, including denial-of-service (DoS) attacks or data corruption. Without proper security controls such as authentication and access restrictions, exposing cluster health data can serve as a reconnaissance tool for malicious actors, increasing the risk of data breaches and operational disruptions.

Figure 6. Cluster information disclosure simulation.

Retrieval of All Indices

When an attacker executes `curl -X GET "http://localhost:9200/_cat/indices?v"` on a vulnerable Elasticsearch instance where X-Pack security is disabled, they gain a detailed map of all indices stored within the cluster.

As shown in Figure 7, this command provides a comprehensive list of details regarding an index named `secret` such as its health status, document counts, storage size, and shard allocation, giving the attacker critical insights into the structure and content of the data.

With this intelligence, an attacker can make a targeted choice of which index to focus on depending on their needs. They may choose to either exfiltrate sensitive information, manipulate records, delete entire datasets, or launch targeted attacks that disrupt the organization’s operations.

Figure 7. Retrieval of all indices simulation.

Reading Data from an Index

When an attacker executes the `curl -X GET "http://localhost:9200/secrets/_search"` command on the vulnerable Elasticsearch instance where X-Pack security is disabled, they can extract and expose highly sensitive information stored within the `secrets` index.

As shown in Figure 8, the `secrets` index contains the three sample credential pairs uploaded by the initialization script upon starting the Elasticsearch Docker instance. This is just one example of what an attacker might find within an index.

Moreover, since Elasticsearch is commonly used as a backend for logging, analytics, and real-time search, organizations often store passwords, API keys, authentication tokens, encryption keys, personally identifiable information (PII), financial records, and confidential business documents within indexed data. Without authentication or access controls, an attacker can retrieve this information with a simple API call, leading to severe security breaches, privilege escalation, and data leaks.

Figure 8. Reading data from index simulation.

Modifying Index Data

As shown in Figure 9, the credential pair `attacker:hacked` was stealthily inserted as another login credential to establish persistence within the system. By injecting credentials into a the `secrets` index, the attacker has effectively created a hidden backdoor account that may allow them to regain access even if initial entry points are patched. As shown in Figure 10, the `curl -X GET "http://localhost:9200/secrets/_search?pretty"` command was used to confirm that the backdoor credentials were indeed appended to the `secrets` index data set.

This tactic enables long-term exploitation, which facilitates data theft, system manipulation, and evasion of detection. The ability for an attacker to modify Elasticsearch index data has a plethora of ramifications. Specifically, attackers could modify security logs or audit trails to erase traces of their presence, making incident response and forensic investigations more challenging.

Figure 9. Modifying index data simulation.

Figure 10. Modifying index data confirmation.

Executing Arbitrary Heavy Queries

The ` _search` query in Elasticsearch is a powerful function allowing users to query and retrieve data from indices. When used with the ` "query": { "match_all": {} }` parameter, it returns every document stored in the cluster.

While this is useful for broad searches, it can be exploited by attackers to overload Elasticsearch and effectively trigger a DoS attack. By repeatedly executing this query on large datasets or across multiple indices, an attacker can force Elasticsearch to consume excessive CPU, memory, and disk I/O, ultimately degrading performance or crashing the cluster.

As shown in Figure 11, a simulated execution of an arbitrary heavy query was conducted but the amount of all the data in this Elasticsearch target was not remotely sufficient to trigger a DoS attack.

Figure 11. Executing arbitrary heavy query simulation.

Dumping Index-Level Configuration

The `_mapping` query and `_settings` query are index-level scoped queries which means they focus on managing individual indices.

The ` _mapping` query defines the schema and structure of documents stored within an index. It specifies field types (e.g., text , keyword , integer, etc.) and data relationships, which are crucial for defining how data is indexed and queried. As shown in Figure 12, the `curl -X GET "http://localhost:9200/_mapping?pretty"` was executed and exposed the entire mapping structure of the sole existing `secrets` index within this Elasticsearch cluster.

The response reveals that the ` secrets` index contains two fields, which are the ` username` and ` password` fields that are stored as text. This may provide potential attackers with a blueprint of how data is structured that enables a more precise attack.

The ` _settings` query manages index-level configurations, such as the number of shards and replicas, refresh intervals, memory settings, and many more critical details. As shown in Figure 13, the `curl -X GET "http://localhost:9200/_settings?pretty"` was executed, which exposed crucial `secrets` index configurations such as the ` number_of_shards` (1) and ` number_of_replicas` (0) that simply indicates a single-shard, non-replicated index.

Furthermore, the routing allocation preference is set to ` data_content`, which indicates tiered data management. Moreover, metadata such as the index `UUID`, `creation date`, and ` version` information are included. All this information may enable potential attackers to identify misconfigurations, performance bottlenecks, and potential attack vectors.

Figure 12. Dumping index-level `_mapping` configuration.

Figure 13. Dumping index-level `_settings` configuration.

Exploiting Script Injection

The `_script` query in Elasticsearch allows dynamic scripting within queries that enables custom computations and advanced filtering on indexed data. While this feature enhances flexibility, it also introduces security risks, in particular, script injection vulnerabilities.

If Elasticsearch is misconfigured to allow untrusted script execution, an attacker can craft malicious scripts to manipulate data, execute unauthorized updates, or escalate privileges. As shown in Figure 14, a script named “payload” was created using the Elasticsearch built-in scripting language called “Painless” to create a field named “ field” in a target document's source and set its value to "hacked".

Subsequently, as shown in Figure 15, the “payload” script was executed on the `secrets` index successfully. Finally, as shown in Figure 16, the `secrets` index data were presented to show that the “payload” scripts have successfully poisoned the index data with the new fields containing the text value “hacked”.

Figure 14. Malicious script creation.

Figure 15. Malicious script execution.

Figure 16. Malicious script injection attack aftermath.

Deleting an Index

When an attacker executes the command `curl -X DELETE "http://localhost:9200/secrets"` on a vulnerable Elasticsearch target where X-Pack security is disabled, they can permanently delete an entire index, which can lead to irreversible data loss, operational disruption, or potentially catastrophic business consequences. Without authentication, authorization, or access control mechanisms in place, anyone with network access to the Elasticsearch instance can execute destructive commands without restriction, making it dangerously easy for an attacker to wipe out critical data with a single API request. As shown in Figure 17, the `secrets` index was effectively deleted in a single `curl` command. Consequently, as shown in Figure 18, when a `_search` query to display the `secrets` index data was executed it failed as the index had already been irreversibly deleted.

Figure 17. Deletion of an index simulation.

Figure 18. Deletion of an index confirmation.

Protection Against Unauthenticated Attack Vectors (X-Pack Enabled)

Exploring the vast array of unauthenticated attack vectors against an Elasticsearch instance with X-Pack disabled was an eye-opening experience that highlights just how exposed a misconfigured deployment can be.

From unauthorized data exfiltration to manipulation of critical indices, the risks were evident. Now, it’s time to shift focus from offense to defense and witness how a simple yet powerful security measure (enabling X-Pack to enforce authentication) can instantly transform an open target into a fortified system, drastically limiting an attacker's ability to exploit vulnerabilities.

To start, the current running Elasticsearch instance will have to be stopped, as shown in Figure 19.

The Elasticsearch instance cannot merely undergo an `elasticsearch.yml` file modification (specifically, to enable X-Pack) and then restart the Docker instance. This is because the `initialization.sh` file will no longer work when X-Pack is enabled as this file contains unauthenticated `curl` commands that will no longer execute successfully due to the lack of credentials.

Besides, the `initialization.sh` file will no longer be of use for this part of the simulation. Therefore, the `elasticsearch.yml` file is first modified (as shown in Figure 20) to enable X-Pack, and then the `docker run` command is executed without all the lines pertaining to the `initialization.sh` file (as shown in Figure 21).

Figure 19. Halting currently running an Elasticsearch Docker instance.

Figure 20. An `elasticsearch.yml` file with X-Pack enabled.

Figure 21. The `docker run` command execution.

With X-Pack enabled, Elasticsearch transforms from an open and vulnerable system into a robust, security-hardened platform.

Authentication mechanisms ensure that only authorized users can access data. As shown in Figure 22, the simple `curl` command to access the Elasticsearch API was essentially blocked due to the lack of authentication credentials as indicated by the JSON output.

It is only when the same `curl` command is integrated with a valid credential pair that the Elasticsearch API will be accessible, as shown in Figure 23.

Figure 22. Failed unauthenticated Elasticsearch API access.

Figure 23. Successfully authenticated Elasticsearch API access.

Another crucial procedure to undertake is upgrading the X-Pack license from ` basic` to ` trial`. As shown in Figure 24, the license status is `basic` by default. Basically, the `trial` license unlocks the full suite of Elasticsearch’s advanced security and enterprise features.

This transition grants access to robust functionalities, such as detailed audit logging, and advanced security measures like field- and document-level access controls.

By enabling the ` trial` license, the following simulations can explore Elasticsearch’s premium capabilities that strengthens their defense against sophisticated threats. As shown in Figure 25, a `curl` command was executed on the `start_trial` API endpoint to convert the license status from `basic` to `trial`.

Finally, to confirm the license status change, the `_license` API endpoint is once again probed to confirm that the status has indeed been converted to `trial`, as shown in Figure 26.

Figure 24. Default X-Pack license status (`basic`).

Figure 25. X-Pack license upgrade command execution (`trial`).

Figure 26. X-Pack upgrade confirmation (`basic` to `trial`).

Authentication Brute-Force via Hydra

Hydra is a powerful and flexible password-cracking tool designed for brute-force attacks against various authentication services and a broad range of protocols, including SSH, FTP, HTTP, RDP, and traditional databases (e.g., MySQL and PostgreSQL) as well as APIs. Hydra automates the process of attempting multiple username and password combinations against login endpoints, making it a common tool for penetration testers and attackers alike.

When targeting an API, Hydra systematically sends authentication requests, exploiting weak or default credentials, misconfigured authentication mechanisms, or lack of rate-limiting protections. If successful, an attacker gains unauthorized access to enable potential exfiltration of sensitive data, modifying records, or escalating privileges.

As shown in Figure 27, this is how an attacker may utilize Hydra to attempt to brute force authentication of an Elasticsearch API using the default username ‘elastic’ and then using a wordlist (e.g., ‘elasticsearc-wordlist.txt’) that contains common Elasticsearch passwords. The attempt eventually failed, which demonstrates the simple but effective hurdle that attackers will have to face to access the Elasticsearch API. As shown in Figure 28, for educational purposes, this demonstrates a successful authentication brute force of the Elasticsearch API using Hydra.

Figure 27. Simulated failed Hydra brute-force attack on the Elasticsearch API.

Figure 28. Simulated successful Hydra brute-force attack on the Elasticsearch API.

Conclusion

Throughout this article, we explored the foundational aspects of Elasticsearch security, with a deep dive into how the X-Pack plugin serves as a crucial layer for authentication. We examined the various unauthenticated attack vectors that arise when X-Pack is disabled, which reveal just how vulnerable an unprotected Elasticsearch instance can be (e.g. The exposure of sensitive data, enabling unauthorized index data modifications, and allowing full index deletion). While enabling X-Pack authentication mitigates these risks, this is just the beginning.

In Part 2, we will shift our focus to the broader security capabilities of X-Pack, including auditing, role-based access control (RBAC), IP filtering, and FIPS 140-2 compliance features. Additionally, we’ll step into a Capture the Flag (CTF) environment to demonstrate how these vulnerabilities can be exploited in real-world scenarios when misconfigurations occur.