What is ElasticSearch?

Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Key features include:

  • Distributed Search: Elasticsearch distributes data and processing across multiple nodes, ensuring high availability and scalability.
  • Full-text Search: It offers powerful full-text search capabilities, including complex search queries.
  • Real-time Indexing: New data is indexed and searchable in near real time, allowing applications to reflect the latest information almost instantly.
  • RESTful API: Elasticsearch's API is RESTful, making it easy to interact with using standard HTTP requests.

What is Kibana?

Kibana is an open-source data visualization and exploration tool designed to work seamlessly with Elasticsearch. It provides a visual interface to:

  • Visualize Elasticsearch Data: Create charts, graphs, and maps to visualize your data.
  • Explore Data: Query and explore the data stored in Elasticsearch using an intuitive interface.
  • Create Dashboards: Combine multiple visualizations into interactive dashboards for monitoring and analysis.
  • Real-time Monitoring: Monitor your data and set up alerts for specific events or conditions.

Prerequisites

Before we start, ensure you have the following:

  • .NET Core SDK installed and running (download from the official .NET website).
  • Elasticsearch installed and running (download from the official Elastic website).
  • Kibana installed and running, configured to point to your Elasticsearch instance.

Setting Up Elasticsearch in .NET Core

At a high level, data flows from your .NET Core application into Elasticsearch for indexing and querying, and Kibana sits on top of Elasticsearch to visualize it:

Step 1: Create a New .NET Core Project

Open a terminal and run:

dotnet new webapi -n ElasticsearchDemo

Step 2: Install Required NuGet Packages

Add the official Elastic client package:

dotnet add package Elastic.Clients.Elasticsearch

Step 3: Configure Elasticsearch

In appsettings.json, add your Elasticsearch URL:

"Elasticsearch": {

  "Url": "http://localhost:9200",

  "DefaultIndex": "products"

}

Step 4: Create an Elasticsearch Service

Create a service class to handle Elasticsearch interactions:

using Elastic.Clients.Elasticsearch;

 

public class ElasticsearchService

{

    private readonly ElasticsearchClient _client;

 

    public ElasticsearchService(IConfiguration configuration)

    {

        var settings = new ElasticsearchClientSettings(

            new Uri(configuration["Elasticsearch:Url"]))

            .DefaultIndex(configuration["Elasticsearch:DefaultIndex"]);

 

        _client = new ElasticsearchClient(settings);

    }

 

    public ElasticsearchClient Client => _client;

}

Step 5: Register the Service

Register the service in Program.cs:

builder.Services.AddSingleton<ElasticsearchService>();

Step 6: Index and Query Data from a Controller

Inject the service into a controller to index and search documents:

[ApiController]

[Route("api/[controller]")]

public class ProductsController : ControllerBase

{

    private readonly ElasticsearchService _elasticService;

 

    public ProductsController(ElasticsearchService elasticService)

    {

        _elasticService = elasticService;

    }

 

    [HttpPost]

    public async Task<IActionResult> IndexProduct(Product product)

    {

        var response = await _elasticService.Client

            .IndexAsync(product, idx => idx.Index("products"));

        return Ok(response.Result);

    }

 

    [HttpGet("search")]

    public async Task<IActionResult> Search(string query)

    {

        var response = await _elasticService.Client.SearchAsync<Product>(s => s

            .Index("products")

            .Query(q => q.Match(m => m.Field(f => f.Name).Query(query))));

 

        return Ok(response.Documents);

    }

}

Step 7: Launch Elasticsearch and Kibana

Start Elasticsearch:

./bin/elasticsearch

Start Kibana in a separate terminal, pointing it to your Elasticsearch instance:

./bin/kibana

Then run the .NET Core application:

dotnet run

Verification

  • Send a POST request to /api/products to index a sample document.
  • Send a GET request to /api/products/search?query=... to confirm the document is searchable.
  • Open Kibana at http://localhost:5601 and confirm the products index appears under Stack Management.

Exploring Data with Kibana

Once your data is flowing into Elasticsearch, use Kibana to explore, visualize, and monitor it. Here's the full walkthrough:

Step 1: Access Kibana

Open your browser and navigate to your Kibana instance:

http://localhost:5601

Step 2: Configure an Index Pattern

Go to Stack Management > Index Patterns, then create a pattern that matches your index:

  • Click "Create index pattern".
  • Enter "products*" as the index pattern name.
  • Select the time field (if applicable) and click "Create index pattern".

Step 3: Discover Your Data

Open the Discover tab to browse and filter individual documents in real time:

  • Select the "products" index pattern from the dropdown.
  • Use the search bar to run free-text or KQL (Kibana Query Language) queries.
  • Apply filters on specific fields to narrow down results.

Step 4: Build Visualizations

Navigate to Visualize Library and create charts from your indexed data:

  • Choose a visualization type — bar, line, pie, data table, and more.
  • Select the "products" index pattern as the data source.
  • Configure the metrics (Y-axis) and buckets (X-axis) to aggregate your data.
  • Save the visualization with a descriptive name.

Example: a pie chart visualization built from indexed product data, broken down by category:

Step 5: Create a Dashboard

Combine multiple visualizations into a single Dashboard for at-a-glance monitoring:

  • Go to Dashboard and click "Create dashboard".
  • Click "Add from library" and select your saved visualizations.
  • Arrange and resize panels, then save the dashboard.

Step 6: Set Up Alerts

Use Kibana's alerting features to get notified when specific conditions are met:

  • Go to Stack Management > Rules and Connectors.
  • Create a new rule (e.g., threshold alert on a metric).
  • Configure a connector — email, Slack, or webhook — to receive notifications.
  • Save and enable the rule.

Conclusion

Integrating Elasticsearch and Kibana into a .NET Core application provides powerful, real-time search and visualization capabilities. With Elasticsearch handling fast, scalable full-text search and Kibana turning that data into interactive dashboards, teams can build applications that are both performant and easy to monitor. Following the steps above should give you a solid foundation to extend into your own production workloads.

 

Recent Posts