Author Archives: Warren Kahn

Coming soon: SCOM as a Service!

2 Replies
See the source image

Let me start by saying that this news is really exciting, as any one in the monitoring world can tell you SCOM has faced a little bit of uncertainty in the past and the announcement of an upcoming SCOMaaS offering from Microsoft sends a clear message that the product has a place in the companies future.

Not a lot of information is available yet but here’s what we do know:

For starters the solution will be containerized which will leverage all of the benefits of containers such as speed of deployment and scaling to name a few.

A SCOM administrator will be able to “lift & shift” their existing SCOM environment into Azure – Aakash Basavaraj Program Manager for the SCOM Team at Microsoft.

This bodes well for existing SCOM customers as it means that the ability to easily migrate to a SCOMaaS solution will be available and that they won’t have to set up their new platform from scratch. This really shows that Microsoft has given thought and care towards truly bringing SCOM to the cloud.

When will is be available?

Unfortunately no details are available yet around timelines or pricing, I know I’ll be watching this develop with keen interest. One thing is for certain SCOM and Azure Monitor are now more firmly hand in hand then ever before.

SCOM 2019: Updated SQL Management Packs for SSRS and SSAS v7.0.22.0

Leave a reply

Updated management packs for SSRS and SSAS have been released, they are available at the below links and support versions from 2012 through to 2019 and now include Power BI Report Server:

SQL Server Analysis Services

SQL Server Reporting Services

SQL Server Analysis Services MP Changelog

What’s New

  • Added support for SQL Server Analysis Services 2012, 2014, and 2016 in addition to previously supported 2017 and up
  • Added Start/Stop Analysis Services Windows Service tasks
  • Implemented Database Status monitor 
  • Updated display strings

SQL Server Reporting Services MP Changelog

What’s New

  • Added support for monitoring SQL Server Reporting Services 2012, 2014, and 2016 in addition to 2017 and up
  • Added Start/Stop Reporting Services Windows Service tasks
  • Updated Event Log Collection Target Management Server Discovery to make it use default SCOM action profile instead of SQL MP Discovery run as profile
  • Updated display name of SSRS Deployment object to display AG name as part of it instead of GUID
  • Updated logic of installation detection for local Reporting Services instance to query Windows Registry instead of WMI
  • Improved error handling for cases when error “Process with an Id is not running” is returned
  • Updated display strings

Bug Fixes

  • Fixed false alerting with status code 400 in monitor Report Manager Accessible for SSRS 2016 and PBIRS
  • Fixed alert parameter replacement failure in monitor Report Manager Accessible
  • Fixed issue with accessing DBConnectionString property of PBIRS
  • Fixed NullReferenceException error when Report Server portal being configured to have several ports

Scheduling reports using your Azure Monitor data

1 Reply

It’s quite a common ask, as to how to take the data in an Azure Monitor Logs workspace and create a report that can then be scheduled. Lets take a look at how we can achieve that.

Immediately we are talking about automation when we use the word schedule and Azure has several tools which we can use. The best fit in this case is Logic Apps.

In this example we will create a report showing if any agents haven’t had a successful heartbeat in the past 24 hours.

Navigate to the Logic Apps blade in your Azure Portal and click +Add

Populate the fields selecting my subscription and resource group, creating a new RG if necessary. Then I’ll give my logic app a name and choose my Azure region and click Review + Create and then Create

Once the deployment is complete I can navigate to the resource and it automatically opens the Logic Apps Designer. Now every Logic App needs to start with a trigger and because I want to run a schedule I am going to use Recurrence

I want this on a daily basis so I’m going to enter 24 Hour as my parameters and then click new step

Search for Run Query and Visualize Results as this option will allow a KQL query as a parameter and the results can be manipulated in a variety of ways. Make sure to select the one for Azure Monitor Logs and not for Azure Data Explorer.

You will need to sign in to create a connection with Azure Monitor Logs. Now populate the fields choosing the subscription, resource group and workspace that contains the data you want to use in your report. Put your query in the relevant section and choose chart type HTML.

Heartbeat
| project TimeGenerated, Computer
| where TimeGenerated < now()
| summarize ["Last Heartbeat"]=max(TimeGenerated) by Computer
| where ["Last Heartbeat"] < ago(24h)

The last thing that needs to be done is connect the logic app to a step to send email, click New Step, search for send an email, I’ll be using Office365 but you can use other providers. Select Send an email v2 and sign in to create the connection.

Populate as below making sure to include the attachments

Click save and we’re all good to go. Now test by clicking Run.

You should receive an email with an attachment and voila opening it will have a nice html table with our query results.

These steps can be easily replicated and amended to be used for any number of handy reports using your Azure Monitor Logs data.

SCOM 2016: UR9 released!

1 Reply

The latest Update Rollup for SCOM 2016 is out, you can get it here

A decent update fixing a couple of issues that have been around for a while. As always test before applying to your production environments.

What’s new and fixed?

  • Fixed: On Windows Computer View of SCOM Console, Queries are optimized for better performance.
  • Fixed: We have updated SPROC to handle Alert Source Path Names which are greater than 512 in length. It was a mismatch where the table which was populating the DB was allowing larger strings while the table which was populated in this view took only 512. This caused data truncation and subsequently console crash.
  • Fixed: Made changes to the IP address parameter to support both IPv4 and IPv6 windows cluster.
  • Fixed: Fixed the issue with conversion of data smaller than 0.01 which was generated by Rule or Monitor. The data was transformed to a wrong (big) value on systems with OS locale language which has comma (“,”) instead of dot (“.”) as decimal format.
  • Fixed: Fixed the schedule reports through the Schedule Management Wizard in APM AppAdvisor. Previously, it was failing with XML Malformed exception.
  • Fixed: In a scenario where SCOM monitors 100s of virtual machines hosted on a single Hyper-v server; every hour the MonitoringHost.exe of each Virtual machine write into the VM page file simultaneously. Due to this concurrent paging, every hour disk I/O increases and database becomes unresponsive. HealthService.exe now have Memory Trimming enabled by default on an hourly schedule. A registry key is provided to disable the memory trimming and control the duration.

Registry key is: “HKLM\Software\Microsoft\Microsoft Operations Manager\3.0\Setup\MonitoringHost\MemoryTrimming“Enable – 0 (Trimming is disabled); 1 (trimming is enabled)DelayInSeconds – Time period agent waits to start trimming (default is 120s)PeriodInSeconds – Recurring period at which the working set should be  trimmed (default is 3600s)

  • Fixed: Downtime duration will be calculated correctly for non-US time format in Downtime Report.
  • Fixed: fn_MPVersionDependentId creates an ID for each installed MP. The ID is a SHA1 hash of string “MPName, Version, Schema”. When the “Schema” part is omitted, it generates different hash which doesn’t match with the expected value. In the fix, “Schema=2” part is re-added in fn_MPVersionDependentId.
  • Fixed: We have made sure that correct end date for Availability, Health and Performance Details are shown in reports
  • Enabled support for MSOLEDBSQL driver so that users may move from SQL Native Client.

Management Pack Recap – January & February 2020 Wave

1 Reply

This is a summary of the wave of Management Packs that were released in January and February 2020. Information and download location in the links provided:

Certificate MP v1.4.3.0 – here
SQL Management Pack v7.0.20.0 – here
Oracle Management pack v1.3.1683.0 – here

If you know of any other Management Packs that have been released recently that I may have missed leave me a note in the comments and I’ll add them

Azure Monitor and SCOM: Together in a hybrid world

Leave a reply

I recently published an article on LinkedIn with my thoughts on monitoring Hybrid cloud with Azure Monitor and SCOM, I am cross posting on my blog for my record but if you prefer you can also read it here https://lnkd.in/d8PvWsH

With the shift of the IT landscape towards public cloud and then toward hybrid there has been some confusion as to which monitoring solutions are the best fit. People with System Center Operations Manager (SCOM) are wondering about the product life-cycle and others are wondering where Azure Monitor comes into the picture.

Hopefully I can shed a little light onto this puzzle.

No alt text provided for this image

First lets look at SCOM, well good news, it’s not going anywhere for quite some time. The current iteration 2019 has a commitment of 10 years of support from Microsoft with an Update Rollup release every 6 months. If you have any on-premise real estate then SCOM is still a strong contender to manage those systems with it’s powerful proven track record, extensible custom monitoring and data-center focused features. If you have extended your network into the cloud you can also leverage these features for your data center-like IaaS components such as VMs.

No alt text provided for this image

Now for your cloud needs, let’s talk Azure Monitor. You may be thinking “but I’m not using Azure” and that’s ok because have I got more good news for you, Azure Monitor can integrate with VM’s running on other clouds, additionally with Azure Arc you can now make use of Azure services and features on any cloud.

No alt text provided for this image

Azure Monitor is a cloud first monitoring tool as such it’s designed to manage and monitor a wide range of cloud components and services leveraging not only the real-time capabilities of Azure Metrics but also the power of Log Analytics both of which feed into a wide variety of insights, visuals and analysis tools. This allows you to have a complete in-depth picture of your cloud real estate from a single location. Azure Monitor is also designed to be configured to monitor only the items you deem most critical which, when done properly results in a lean noise free monitoring experience.

No alt text provided for this image

So what should I use to monitor my Hybrid Cloud scenario?

The short answer is: No matter where you are you are on your cloud journey there’s a monitoring scenario that has you covered.

Regardless of whether you are migrating your VMs to an IaaS platform; refactoring your on-premise applications to leverage PaaS/SaaS services or have some applications which reach back into your on-premise environment, you have options and you don’t have to use a single tool either

Lets unpack this a bit.

If you have a large on-premises environment then certainly a data center designed tool such as SCOM will meet your needs and it can also cater for any VMs that you have migrated into the cloud. This will of course give you the on-prem monitoring experience that you are familiar with, which does offer a level of comfort while preparing to move to more cloud focused services.

Similarly if you are nearing the end of your cloud journey then Azure Monitor has got you covered from Billing to Subscriptions and everything in-between. Not only will you be able to proactively handle your tenant and resources through automation but you will also have access to rich interactive reports in the form of workbooks as well as machine learning capabilities.

However if you are between these two scenarios you can use both tools, in fact I’d even encourage it, they work very well together and leverage each others strengths to give a comprehensive single pane of glass across any Hybrid environment.

No alt text provided for this image

Silect Dashboards for SCOM

1 Reply

Silect has released an awesome set of dashboards for SCOM and best of all they’re free. You can request your copy here.

All you need to get started is PowerBI Desktop available here, then load the pbix file and edit the data source to point at your SCOM database instances.

Then hit refresh and you’ll have 10 nice health dashboards with an overview of your SCOM environment.

I really like this solution, it’s elegant and easy to setup. Great job Silect.

SCOM 2019: UR1 has been released!

3 Replies

The much anticipated release of UR1 for SCOM 2019 is here and you can get it from the update catalog.

There’s quite a bit to unpack here so lets have a look

The big changes are of course support for gMSA which I for one have been looking forward to for a while. The security benefits are obvious and I can say while it does work well be aware that it is complex to implement so read those instructions thoroughly.

The one click upgrade experience is also a step in the right direction making things much simpler for the SCOM admin, just note it only includes the server update so you will still need to upgrade your console, reporting etc manually.

See below for a complete list of all the changes

Improvements and issues that are fixed

Web Console Fixes/Changes:

  • State widget now supports sorting by health and age. 
  • Alert widgets can now be searched on Date Time and sorted based upon age and severity.
  • The alert link in email notification when browsed throws an error, “Your session with the Web Console server expired” even though the user was not logged in web console . You will now see the login page post this fix.
  • Alert summary view in SCOM Web console was fixed in size.  This can be expanded as required now.
  • When Alert state is changed to some custom state, then these alerts are not displayed in web console. Alerts with custom resolution states will be displayed now.
  • Few additional scrollbars appear when a customer widget is created in web console or browser window size is reduced.
  • Improvement: SCOM views will load/save much faster than before. 

Unix/Linux/Network Monitoring Fixes/Changes:

  • SCX by default does Info level logging which causes SCX logs to be filled up soon. With 2019 UR1, Warnings and Errors will be logged by default not Info.
  • In the network device monitoring, if the node property is changed to null/empty, it displays the old value of the property not the null value. The actual value will be displayed now which includes null.
  • In a scenario where multiple (~500) virtual network interfaces are running on agent, SCOM sometimes received heartbeat failure alerts for such agents. SCOM will monitor only physical interfaces hosted on these machines. For monitoring virtual interfaces, you need to set “enumvif” to true. 
  • SNMP GET or SNMP WALK resulted monitoring host to crash when the network object ID is equal to ULONG Max. There is an error with event ID 4000 due to this crash.
  • For linux distros servers, when the kernel version >= 4.18, then file system related performance data is shown as 0 in SCOM. This is fixed in FS provider to make it kernel version agnostic and collect file system stat info.
  • Use of Async Windows Management Infrastructure (MI) APIs is default functionality from SCOM 2019 UR1 for scalability improvements.More details here. 

 Admin Console Fixes:

  • When admin created “one-time maintenance mode schedule” for non-English locales, SCOM console displays an error “The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection”. Admin would be able to create one-time maintenance schedule now.
  • When SCOM agent is multihomed, then SCOM 2019 MS always displays logon type as “Interactive” for this agent. The correct logon type will be displayed now.
  • Unit monitor to check correct logon type for Run as Account had incorrect name and Operational State.
  • “Operations Manager Products” view in Admin console did not update the “Version” column for the installed component version. This column will now reflect the updated version of all the components listed.

Others:

  • When system proxy is configured on the Reporting Server; proxy overrides are ignored by the “Report Console Watcher Monitor” and “Web Console Watcher Monitor” and as a result HTTP/HTTPS access fails.  With UR1 fix, “Set proxy Direct” parameter of the mentioned monitors can be set to true and URLs will be accessed.
  • SCOM did not discover Windows Cluster which are deployed on servers with IPV6 network only enabled. IPV6 network will be supported for cluster monitoring now.
  • When the Domain controller is moved to a different AD site, then SCOM does not display the correct site name. Any dynamic group which are created for SiteName will also fail

 In addition to these, all the issues fixed in SCOM 2016 UR8 and prior UR for SCOM 2016 are also fixed in SCOM 2019 UR1. Details of the fixes are below.

  • In a scenario where SCOM monitors 100s of virtual machines hosted on a single Hyper-v server; every hour the healthservice.exe of each Virtual machine write into the VM page file simultaneously. Due to this concurrent paging, every hour disk I/O increases and database becomes unresponsive. HealthService.exe now have Memory Trimming enabled by default on an hourly schedule. A registry key is provided to disable the memory trimming and control the duration.

          Registry key is: “HKLM\Software\Microsoft\Microsoft Operations Manager\3.0\Setup\MemoryTrimming”

         Enable – 0 (Trimming is disabled); 1 (trimming is enabled)

         DelayInSeconds – Time period agent waits to start trimming (default is 120s)

         PeriodInSeconds – Recurring period at which the working set should be trimmed (default is 3600s)

  • Historical data do not appear, if input reporting end time is before group creation time. With this fix, historic data for a group (if data is available for objects in the group) would be displayed irrespective of group creation time.
  • Maintenance mode state changes which are recorded in MaintenanceModeStage table requires grooming when table grows. If the table is large, grooming takes longer and the operation times out with SQLTimeOut exception.
  • If a group is renamed in a Management pack, then console shows the new value but Powershell command Get-SCOMGroup returns the old name of group. Database Updates functionality was inconsistent for SCOM group renaming through MP and SCOM Console.
  • CPU Spike issues because of workflows running on all agents at the same time is addressed through script optimization and removing the sync time. 
  • Improvement: Sometimes SQL stored procedure “p_SelectForNewTypeCache” takes long time to complete, and SDK service fails to start. This is fixed and above SQL stored procedure will complete faster now.
  • Improved the performance of SCOM console in listing the groups.
  • Users of a scoped group are not able to use the Console.
  • SCOM console crashes while trying to connect to Azure Log Analytics and Azure Monitor.
  • SCOM Network Device Re-Discovery now probes for SNMP V3 devices too.
  • Agents by Health State report shows duplicate agent names.
  • Fixed an issue that prevented addition of a group in the Storage Spaces Direct 2016 management pack dashboard.
  • Linux agent is not able to get the correct version and port details for JBoss EAP 7.1.
  • An issue that lead to creation of multiple empty temp files in the /tmp directory of Linux servers has been fixed.
  • Fixed the formatting issue with the output for the task ‘Top10 CPU Processes’ when using Windows Management Infrastructure (MI) APIs.
  • Fixed an issue that caused the corruption of /etc/login.cfg file on AIX 7 machines during install/upgrade of the agent.
  • AIX Agent is now transitioned to 64-bit package to accommodate more stack and heap space if needed to avoid any stack/heap overflow which occasionally leads to heartbeat failure.
  • Free memory calculation accommodated appropriately on RHEL-7 platform.

SCOM: OpsLogix has updated their Oracle Management pack v1.3.1683.0

1 Reply

The great crew over at OpsLogix has a new Oracle MP for us to ring in 2020, it’s available here

What’s different?

Changes

  • Rewritten the complete backend to make usage of a collector service. This way we detach the SCOM modules from the Oracle client resulting in a way less footprint and more performance.
  • Moved the Oracle configuration dashboard from monitoring pane to Administration pane.
  • Rewritten the Oracle configuration dashboard. Please read documentation how to use.
  • Added Opslogix General Reports MP to the install package.
  • The container DB discovery is now by default disabled.

Fixes

  • The connection monitor now only reports an error when it is x times (2 default) in unhealthy state.
  • When creating a connection alert a parameter could not be resolved for the alert description.
  • During getting sysstats information with containing very big numbers the converting to SCOM data could be broken

and yes it is upgradable from  V1.3.0.0 or later

SCOM: Updated SQL Management Pack v7.0.20.0

1 Reply

A new version of the sql agnistoc MP is out get it here

What’s New

  • Updated MP to support SQL Server 2019 RTM
  • Added filter by edition to “Local DB Engine Discovery”
  • Redesigned DB Space monitoring to improve performance: Enabled by default
  • monitors and performance rules targeting Database which watch for disk space consumption by ROWS Filegroups and Logfiles
  • Redesigned DB Space monitoring: Added two monitors and two performance rules
  • targeting Database to watch for disk space consumption by In-Memory and
  • FILESTREAM data
  • Redesigned DB Space monitoring: Read-only filegroups now count as well
  • Redesigned DB Space monitoring: Disabled by default all workflows targeting
  • Filegroups, Files, Logfiles
  • Redesigned XTP performance counters to make them completely version-agnostic
  • Added attribute “TCP Port” to “SQL DB Engine Class” and updated “DB Engine
  • Discovery” to populate the new property
  • Added summary dashboard for SCOM 2019 Web Console (HTML5)
  • Added support for cluster nodes with disjoined namespaces
  • Added sampling to algorithm of monitor “WMI Health State” in order to eliminate
  • false alerting on cluster SQL Server instances
  • Updated alert descriptions of monitors “Availability Database,” “Availability Replica,”
  • and “Availability Group” (generating alerts still disabled by default)
  • Updated monitor “Product Version Compliance” with versions of most recent public
  • updates to SQL Server
  • Disabled by default monitor “Buffer Cache Hit Ratio” and changed its threshold
  • from 0% to 90%
  • Disabled by default monitor “Page Life Expectancy”
  • Removed monitors “Availability Database Join State” and “Availability Replica Join
  • State” as not useful
  • Updated display strings
  • Revised columns on DB Engine state views

Bug Fixes

  • Fixed: monitor “Service Principal Name Configuration Status” raises false alerts
  • because of case-sensitive comparison
  • Fixed: “Local DB Engine Discovery” crashes when Windows has Turkish locale
  • Fixed issue that caused performance degradation in workflows “General Always On
  • Discovery,” “Database Replica Discovery,” and “Always On System Policy
  • Monitoring”
  • Fixed: “General Always On Discovery” throws errors on environments with several
  • Distributed Availability Groups
  • Fixed monitoring issue in case of Database is replicated by Always On Availability
  • Group
  • Fixed empty property bag when Availability Group has cluster type NONE
  • Fixed wrong target in alerting rule “DB Backup Failed to Complete”
  • Fixed rule “MSSQL Integration Services on Windows: The package restarted from
  • checkpoint file” and its alert
  • Fixed rule “OS Error occurred while performing I/O on pages“ and its alert
  • Fixed: “DB Disk Write Latency” and “DB Disk Read Latency” monitors and
  • performance rules get wrong performance metric
  • Fixed alert description of monitor “WMI Health State”