Skip to main content
Version: Atlas v5.1

Using Data Hub

Data Hub gives Splunk users visibility into their data and the ability to add clarity by creating Data Definitions when their role allows it. These definitions help teams apply stronger change management discipline to their Splunk environment. Paired with this capability, Data Hub also provides visibility into dataset utilization so Splunk Admins can better understand what data is being ingested, who owns it, and how it is being used.

For setup details, see Configuring Data Hub.

Data Hub expands macros, eventtypes, tags, and wildcard-based search references when analyzing utilization so the reported activity better reflects the effective search logic behind dataset usage.

Following these steps will help Splunk admins regain control of their Splunk ecosystem and control data ingests.

  1. Ensure utilization is being tracked by configuring Data Hub.
  2. Set ownership of Indexes using bulk apply.
  3. Perform utilization analysis on your high ingest indexes to ensure they are being effectively used.
  4. Create and assign data labels such as 'Networking' or 'OS Security' to group together similar indexes.
  5. Perform routine reviews on ingests and ensure all data flowing into Splunk is owned.

Data Hub Page​

Data Hub is designed to help teams manage dataset lifecycle tasks with fewer context switches. The main data table enables users to investigate their data effectively. Review the below features to fully understand the capability offered by Data Hub.

Data Set Type​

Use the Data Set Type input in the page banner to control the granularity of datasets shown in the table. This selection changes how Atlas groups the data and therefore what each table row represents.

  • Index: Shows one dataset for each index.
  • Index-Sourcetype: Shows datasets for each sourcetype within an index.
  • Index-Sourcetype-Source: Shows datasets for each source within its index and sourcetype.

Default Views & Additional Columns​

By selecting the View dropdown, a user can select a pre-made collection of columns for the table to easily get started in Data Hub. The currently selected view is preserved on page reload. The available views include:

  • Business Unit: Organizes datasets by Business Unit to help teams review related data together.
  • Data Label: Organizes datasets by their assigned Data Labels for label-based analysis.
  • Inventory: Focuses on Data Label, Owner, Business Unit, Contact information, and ingest details for each dataset.
  • Ownership: Organizes datasets by Data Owner to support ownership and accountability reviews.
  • Retention: Focuses on data retention information and reveals indexes hitting size limits. Only when Data Hub is in Index view.
  • Utilization: Focuses on the utilization activity of a data set and compares ingest to overall usage.
  • Custom: Represents a user-defined column selection.

Selecting a default view clears any active grouping and applies that view's intended organization. Selecting the data table's column button enables the user to add additional columns to the view. Your column selection is saved, but is cleared by selecting a default view.

Inline Editing​

Atlas Creators and Atlas Admins can double click on columns marked with a pencil icon and update fields without opening any modals. Atlas Viewers can review these values but cannot edit them.

Bulk Actions​

Atlas Creators and Atlas Admins can select dataset checkboxes to reveal bulk actions. Atlas Viewers can review datasets and utilization results, but cannot use bulk actions.

Define Datasets​

Select one or more datasets, then select Define Datasets to open the Bulk Update modal. Use this modal to apply Data Definition information to more than one dataset at a time. Any field left blank is not updated, changed, or cleared. New information entered in the modal overwrites the existing value for the selected datasets.

Email Report​

Select one or more datasets, then select Email to open the Email Report modal. Use this workflow to send a report and status update for the selected datasets.

  1. Add one or more recipients in Email Contact. These recipients receive a report containing all selected datasets.
  2. Optionally enable Email Dataset Owner(s) to send each owner a separate report containing only the datasets they own.
  3. Review or update the prefilled Subject and Body fields.
  4. Review the Selected Dataset(s) table before sending. The table includes Dataset Name, Utilization/GB, Ingest (GB), Total Utilization, Owner, and Business Unit.
  5. Select Send Emails to send the report. Atlas includes the selected dataset information as a CSV attachment.

Datasets without an owner are included in reports sent to the Email Contact recipients but do not receive a separate owner report.

Detailed Modals & Actions​

Selecting the kebab (three dots) button on the far right of the table enables the user to view additional details and perform actions for supported workflows.

  • Show Data In Splunk: Opens up and runs a search against the dataset for the last 24 hours.
  • View Dataset Info: Opens the details modal on the data definition page. Atlas Admins and Atlas Creators can update the definition in more detail here.
  • View Dataset Utilization: Reveals the utilization of the dataset. Users can review tracked SPL, drill into linked knowledge objects, and navigate directly to related dashboards or searches from this modal.

Utilization Investigation​

The Utilization view helps teams understand how datasets are being used across searches and dashboards. From the utilization modal, users can inspect the tracked activity, review the SPL associated with that usage, and navigate directly to the related object when it is available to them. In search-peered environments, Data Hub resolves dashboard links to the appropriate destination so users can continue their investigation in context.

Utilization tracking is designed to account for expanded search content such as macros, eventtypes, tags, and wildcard references so the reported activity better matches what users actually run in Splunk.