This posting is a continuation of my basics series, this one deals with Understanding Grooming and Aggregation, two very important concepts which can have great impact on your environment if they are not functioning properly.
First you need to understand the different between these two concepts, at their most basic, Grooming is the method where data older then a specific period, called the retention period, is deleted. Aggregation is the method where raw data, which is collected on short intervals i.e. 5 minutes, are averaged into hourly and daily time intervals used for reporting purposes.
The Operations Database
A stored procedure, called p_PartitioningandGrooming is run by a SCOM rule on a once a day schedule. The retention settings for each data type can be set in the console under Administration > Settings > Database Grooming.
You can see the history of the OpsDB grooming jobs with the following SQL query:
select * from InternalJobHistory order by InternalJobHistoryId desc
A Status of 1 indicates a successful completion.
The Operations Data-warehouse
The Data Warehouse is more complex, you cannot change data retention settings from the console, they exist in a table called StandardDatasetAggregation, which contains the grooming and aggregation retention periods, intervals and other related data.
Data in the DW database moves through several tables as part of the aggregations process as the data is collected it is written to a Raw Data table, from there is will move to a Staging table where it will be aggregated and then moved to a relevant aggregated data table (Hourly / Daily) where it will be available to the reporting services.
Then once the data in these tables is older then the data retention period it will be groomed out of the database.
So why is it important for aggregation to work?
Reports use the hourly and daily data in order to generate, this means that if your aggregations are not working or slow there will be gaps in your reporting. Example below:
And there you have it. the basics behind aggregation and grooming, I hope this has been informative.
