Author Archives: Warren Kahn

Kusto Detective Agency Season 2: Case 2 – Catch the Phishermen!

Challenges

Click for challenges

Onboarding: Here
Challenge 1: Here
Challenge 2: This article
Challenge 3: Here
Challenge 4: Here
Challenge 5: Here
Challenge 6: Here
Challenge 7: Here
Challenge 8: Here
Challenge 9: Coming soon
Challenge 10: Coming soon

Time to catch some phishermen! I’m sure we have a special dislike for unsolicited phone calls, worse if it’s a potential scammer! Yet another great real-world use case for the power of KQL, I’m sure my Cybersecurity colleagues had a great time with this one.

General advice

For this case there are some assumptions that have to be made about certain behavior, I would suggest starting as simply as possible as once you go down certain rabbit holes you can go off down a long path which will not get you to the right answer. Check out those clues!

Challenge: Case 2

Case 2 challenge text

Hey Detective,

We’ve got another case that needs your expertise! The people of our city are being targeted by phishermen, and they need your help to stop them in their tracks.

The complaints are pouring in, and people are fed up with the sudden increase in phishing calls attempting to steal their identity details. We can’t let these scammers get away with it, and we need your help to catch them!

The police have asked for our assistance, and we’ve got a massive data set to work with. We’ve got listings of all the calls that have been made during the week, and we need to find the source of the phishing calls.

It’s not going to be easy, but we know you’re up for the challenge! We need you to analyze the data and use your detective skills to find any patterns or clues that could lead us to the source of these calls.

Once we have that information, the police can take action and put a stop to these scammers once and for all! Are you ready to take on this challenge, detective?

We’ve got your back, and we know you can do this! Let’s catch those phishermen!

Best regards,
Captain Samuel Impson.

Time to find that scammer!

Query Hint

The are a couple of different way to get to the right phone number but the command we’ll use are the same. Check out these KQL commands for some help extend, join and dcount.

Depending on the assumptions you’ve made about the scammer behavior it will also affect how you start writing your query.

Solution – Spoilers below

What were your assumptions detective?

Query Case 2

//The key behaviors that come to mind are firstly that the scammer probably called a lot of people, secondly those calls were probably on the short side because we know the citizens are getting fed up.

//Who called the most citizens?

Another criminal brought to justice, great work detectives! This wasn’t the most exciting case but the application of useful KQL commands more than made up for it, this challenge was a great learning exercise, nicely done Kusto Detective Agency team!

Kusto Detective Agency Season 2: Case 1 – To bill or not to bill?

8 Replies

Challenges

Click for challenges

Onboarding: Here
Challenge 1: This article
Challenge 2: Here
Challenge 3: Here
Challenge 4: Here
Challenge 5: Here
Challenge 6: Here
Challenge 7: Here
Challenge 8: Here
Challenge 9: Coming soon
Challenge 10: Coming soon

In this first case we’re asked to solve a billing problem, not the most exciting thing but certainly interesting with some real-world applications for the use of data. I quite enjoyed this challenge as it reminded me to keep things simple and not discount any ideas as silly just yet.

General advice

For this case the wording tripped me up a little bit, make sure you understand what’s being asked and check out the training if necessary. I will say, while I like the idea of the training it eventually put me on the wrong track, so use it but also keep an open mind.

Challenge: Case 1

Case 1 challenge text

Dear Detective,

Welcome to the Kusto Detective Agency! We’re thrilled to have you on board for an exciting new challenge that awaits us. Get ready to put your detective skills to the test as we dive into a perplexing mystery that has struck Digitown.

Imagine this: It’s a fresh new year, and citizens of Digitown are in an uproar. Their water and electricity bills have inexplicably doubled, despite no changes in their consumption. To make matters worse, the upcoming mayoral election amplifies the urgency to resolve this issue promptly.

But fear not, for our esteemed detective agency is on the case, and your expertise is vital to crack this mystery wide open. We need your keen eye and meticulous approach to inspect the telemetry data responsible for billing, unravel any hidden errors, and set things right.

Last year, we successfully served Mayor Gaia Budskott, leaving a lasting impression. Impressed by our work, the city has once again turned to us for assistance, and we cannot afford to disappoint our client.

The city’s billing system utilizes SQL (an interesting choice, to say the least), but fret not, for we have the exported April billing data at your disposal. Additionally, we’ve secured the SQL query used to calculate the overall tax. Your mission is to work your magic with this data and query, bringing us closer to the truth behind this puzzling situation.

Detective, we have complete faith in your abilities, and we are confident that you will rise to the occasion. Your commitment and sharp instincts will be instrumental in solving this enigma.

Sincerely,
Captain Samuel Impson.

Right let’s get down to business and get the citizens of Digitown their correct bills!

Query Hint

There are two things wrong with the billing run this month and you’ll have to find both to get the right answer. KQL commands that will be useful are arg_min and distinct.

There is a bit of an investigation that needs to be done to uncover the issues with the data and there are various angles you can take, such as looking at specific houses or dates just to name a couple.

Solution – Spoilers below

Have you found the two things wrong with the billing?

Query Case 1

//The most obvious issue is that some customers are being double billed, so we need to remove those duplicates. Also, it turns out that some customers are using negative water and electricity, what doesn’t seem possible so let’s get rid of that too.

Great work detectives! This case gave me a nice opportunity to stretch my “KQL legs” and i found it to be a fun experience. I’m definitely looking forward to the next one.

Kusto Detective Agency Season 2 – Onboarding

8 Replies

Challenges

Click for challenges

Onboarding: This article
Challenge 1: Here
Challenge 2: Here
Challenge 3: Here
Challenge 4: Here
Challenge 5: Here
Challenge 6: Here
Challenge 7: Here
Challenge 8: Here
Challenge 9: Coming soon
Challenge 10: Coming soon

It’s exciting to have another season of the Kusto Detective Agency, this is an excellent way to learn KQL and gain some useful skills that are useful with many Microsoft products including Azure Monitor, Sentinel, M365 Defender and Azure Data Explorer (ADX) to name a few.

General advice

If like me, you’re still in full detective mode from last season, then take a moment to reset your “complexity level” we’re starting again with the basics and it’s best to approach these 10 challenges in that way, from simple to complex.

Challenge: Onboarding

Onboarding challenge text

If you have been here for Season 1, you may be surprised to find yourself as a Rookie again. You see, it’s all about innovation and hitting refresh. So, it’s a fresh start for everyone. Yet we believe in excellence and that’s why we need your detective skills to unveil the crème de la crème of detectives from the past year, 2022. This is like the ultimate leaderboard challenge where we crown the “Most Epic Detective of the Year.” Exciting, right?

Imagine our agency as a buzzing beehive, like StackOverflow on steroids. We have a crazy number of cases popping up every day, each with a juicy bounty attached (yes, cold, hard cash!). And guess what? We’ve got thousands of Kusto Detectives scattered across the globe, all itching to pick a case and earn their detective stripes. But here’s the catch: only the first detective to crack the case gets the bounty and major street cred!

So, your mission, should you choose to accept it, is to dig into the vast archives of our system operation logs from the legendary year 2022. You’re on a quest to unearth the absolute legend, the detective with the biggest impact on our business—the one who raked in the most moolah by claiming bounties like a boss!

Feeling a bit rusty or want to level up your Kusto skills? No worries, my friend. We’ve got your back with the “Train Me” section. It’s like a power-up that’ll help you sharpen your Kusto-fu to tackle each case head-on. Oh, and if you stumble upon a mind-boggling case and need a little nudge, the “Hints” are there to save the day!

Now, strap on your detective hat, embrace the thrill, and get ready to rock this investigation. The fate of the “Most Epic Detective of the Year” rests in your hands!

Good luck, rookie, and remember to bring your sense of humor along for this wild ride!

Lieutenant Laughter

To get started we simply need to identify the detective who won the most bounties from season 1, luckily we have everything we need to get started.

Query Hint

There are a couple of key pieces of information we need to solve this:

We know there are different IDs for each detective
Only the first detective with the correct solution can claim the bounty
Who has the most bounties?

KQL commands that will be useful to achieve this are extend, summarize arg_min and join.

Solution – Spoilers below

To solve this, we need to find out the bounty for each case and then join that with the winner of each case.

Query Onboarding

Bonus answer in 4 lines of code

Turns out the detective with the most bounties is also the detective with the most entries. Which just goes to show you, there are different ways to get the right answers.

DetectiveCases
| summarize count() by DetectiveId
| where isnotempty(DetectiveId)
| top 1 by count_

All in all I’m glad season 2 is here and I am excited to crack these cases, good luck detectives and welcome aboard!

Kusto Detective Agency Season 2 is here!

Leave a reply

Welcome back detectives, to a new exciting season of Kusto Detective Agency, this time around there are 10 cases to solve and some new tools to help you sharpen those KQL skills!

What is it?

The Kusto Detective Agency is a set of challenges that is designed to help you learn the Kusto Query Language (KQL), which is the language used by several Azure services including Azure Monitor, Sentinel, M365 Defender and Azure Data Explorer (ADX) to name a few. The challenges are gamified and interactive and consist different exciting cases across two seasons.

Each case has a different scenario that you need to solve using KQL queries, where you can earn badges, and they get progressively more difficult as you help the citizens of Digitown.

Season 1 is still available, and I talk about my experience with those challenges here.

Where can I get started?

It’s easy to get started just creating your free ADX cluster and report for duty at the detective agency!

Access the challenges here – https://detective.kusto.io/
Create your free ADX cluster here – https://aka.ms/kustofree

What’s new?

Hints return from season 1 but the new and exciting feature is a set of training that you can complete to prepare you for each case. This highlights specific commands and techniques that are relevant to solving the various puzzles. Just click “Train me for the case to get started”.

My thoughts

KQL is very valuable considering all of the products that make use of the language and being able to write a basic query does make working with those products much easier. Learning in this gamified way also makes the process more interesting and if the cases from season 2 are anything like season 1 we’re in for a lot of fun. I will be documenting my experience with season 2 and would highly recommend the Kusto detective Agency for anyone who could benefit from KQL skills.

Supercharge your Career Development Plan with a little help from AI!

Leave a reply

Career development planning can be challenging, time consuming and even overwhelming. I’ve looked at using popular AI tools such as ChatGPT to see if this process can be easier and let me tell you, it certainly can be.

Here are several AI prompts and instructions that I’ve used to great effect and wanted to share so that others can benefit from them too.

CareerGPT

First CareerGPT which now comes in two flavors both of which will help you create a quality career development plan quickly and easily, which you can then use in future planning and career discussions.

The guided experience (v1) – Step through an easy to uses guided approach to creating a development plan with explanations and tips to help you get the best outcome.

The prompts, instructions and original article can be found here – https://aka.ms/careergpt

The advanced experience (v2) – Join your virtual career panel that will ask you questions and make recommendations to help you build your career plan. This is a more “human” experience and is focused on the output rather than the guided step-by-step approach.

Learn more about the advanced experience here
The prompts and instructions can be found here – https://aka.ms/CareerGPTv2
Check out my colleague Werner building his career plan, in under 10 minutes, using the guided experience over on Youtube

Role advisor

Sometimes all we need to get started are some ideas for future roles, this conversation can unfortunatly be a little bit like asking a child “What do you want to be when you grow up?” If all we’ve even been exposed to are doctors, teachers and tiktokers then it’s hard to think outside that box.

Enter Role Advisor here to help you find career options suited to your skillset, read more about it here

The prompts and instructions can be found here – https://aka.ms/RoleAdvisor

Feedback

Hopefully this has been useful, it would be awesome if you could take 1 minute to fill out a short survey to let me know how this worked for you, so that I can make this better in the future!

Find the survey here – aka.ms/careergptfeedback

Enjoy and happy career planning!

SCOM 2019: Update Rollup 5 is out!

Leave a reply

UR5 for SCOM 2019 is available get it here!

A reasonable update with some quality-of-life fixes and enhancements, the most notable addition is the ability to discover SCOM MI instances in Azure which supports the hybrid approach to giving a single pane of glass across your on-prem and cloud IaaS estate.

Improvements

Discover Azure Monitor SCOM Managed Instance (preview) from SCOM console.

Issues that are fixed

Operations Manager 2019 Update Rollup 5 includes fixes for the following issues:

Fixed an issue where editing an existing Maintenance Mode schedule does not change the Reason and/or Comment.
Fixed an issue where when setting Maintenance Mode via PowerShell, the Availability Reports were not reflecting correct information.
Fixed the issue in which Member column in group View was introducing delay in group operations.
Fixed an issue of users getting HTTP200 error when trying to setup Log Analytics connection.
The script (GetOpsMgrDBPercentFreeSpace.vbs) which is part of System Center Core Monitoring MP monitor has been moved from VBS to PowerShell and, now reports Operations DataBase free space correctly.
A new registry key (for debugging purposes) to enable Bad.xml file creation is introduced in UR5 which does not exist by default but needs to be created. Registry key details below.
- [HKLM\SYSTEM\CurrentControlSet\Services\HealthService\Parameters] – XmlDebugEnabled (DWORD) – default 0 and 1 for ENABLED

Security Enhancements

Fixed multiple Web Console Security Vulnerabilities.
- Note: The Web.config files of both HTMLDashboard and MonitoringView web apps will be replaced, so any changes done to settings inside of these will have to be remade.
The organization of temporary files used for kerberos based authentication is further enhanced to prevent any misuse.
Fixed Data parsing issues in Linux agent that might cause the agent to crash.

Unix/Linux/Network monitoring fixes and changes

Fixed an issue where msgAuthenticationParameters needs to have 0 length during engine discovery of SNMPv3 devices. Also
Fixed an issue related to SNMP Discovery where we see MonitoringHost.exe crashes.
Fixed the issue where user was unable to run Get-SCXAgent and Invoke-SCXDiscovery remotely using Invoke-Command.
Fixed Linux agent crash issue caused by variable out of scope issue for _HandleGetClassReq.
Fixed an issue that would causes Linux agent to crash when DSC provider is installed.
Added supportability on Operations Manager Linux Agent for Rocky Linux 8, Alma Linux 8. systems with OpenSSL 3.0, RHEL 9.0 and Ubuntu 22.

Monitor better, react faster!

Leave a reply

“Perception is the key to reaction; the sharper your perception, the quicker your reaction” – Unknown

This adage holds true in many aspects of life, including cloud monitoring and security. In today’s digital world, cloud infrastructure is the backbone of most organizations, and ensuring the security and availability of these resources is critical. To do this effectively, you need to have a clear view of your cloud infrastructure, and be able to detect and react to threats quickly. This is where perception comes in.

Cloud monitoring is the process of tracking and analyzing the performance, availability, and security of cloud resources. It involves collecting data from various sources and analyzing it to identify trends, anomalies, and potential threats. A key aspect of effective cloud monitoring is having a sharp perception of what’s happening in your cloud environment. This means being able to see and understand the data that’s being generated by your cloud infrastructure, and being able to quickly detect any anomalies or deviations from normal behavior.

One of the biggest challenges with cloud monitoring is the sheer volume of data that’s generated by modern cloud environments. With thousands of resources spread across multiple regions and availability zones, it can be difficult to get a clear view of what’s happening in your cloud environment. This is where cloud monitoring tools such as Azure Monitor and Sentinel come in. These tools are designed to help you collect, analyze, and visualize cloud data in a way that’s easy to understand and act upon.

However, even with the best cloud monitoring tools, perception is still key. You need to be able to interpret the data that’s being generated by these tools and make quick decisions based on that information. This requires not just technical expertise, but also the ability to understand the context and significance of the data that’s being generated.

Cloud security is another area where perception is critical. With cloud environments, security is not just about protecting physical assets; it’s also about protecting data and applications. This means being able to detect and react to threats quickly, before they can cause significant damage. Again, having a sharp perception of what’s happening in your cloud environment is essential for effective security. This also includes tracking and analyzing security events in your cloud infrastructure, such as unauthorized access attempts, data breaches, and malware infections. It requires collecting and analyzing large amounts of security data, and being able to quickly identify and respond to security incidents. This requires not just technical expertise, but also the ability to quickly interpret and understand the significance of security events.

In conclusion, perception is the key to effective cloud monitoring and security. The sharper your perception, the quicker your reaction, and the more effectively you can protect your cloud infrastructure. To achieve this, you need to have the right cloud monitoring tools in place, as well as the expertise to interpret and act on the data that’s being generated. With the right approach, you can ensure the security and availability of your cloud resources and keep your organization safe from cyber threats.

https://www.linkedin.com/pulse/monitor-better-react-faster-warren-kahn

Azure Monitor Basics: Best practices for configuring Azure Monitor alerts

2 Replies

Azure Monitor is a powerful tool that can help you keep track of the performance and health of your Azure resources. One of its most useful features is the ability to set up alerts that notify you when certain conditions are met. However, in order to make the most of this feature, it’s important to follow some best practices when configuring your alerts.

Be specific with your alerts: When setting up alerts, it’s important to be as specific as possible. This means identifying the exact resource or metric that you want to monitor, as well as the specific condition that should trigger the alert. For example, instead of setting up a general alert for “high CPU usage,” set up an alert specifically for “CPU usage on WebApp1 exceeds 80% for 15 minutes.”
Use alert suppression: In some cases, you may not want to receive alerts for certain conditions. For example, you may want to suppress alerts during maintenance periods or when you know that a particular resource is experiencing high load. Azure Monitor allows you to suppress alerts based on specific conditions, such as time of day or the presence of specific keywords in the alert description. For example, you can suppress alerts during non-business hours by setting the suppression time to outside of your business hours.
Use action groups: Azure Monitor alerts can be configured to take a number of different actions when triggered, such as sending an email, creating a ticket in a service management system or even triggering an automation runbook. To make the most of this feature, it’s a good idea to create action groups that group together different actions for different types of alerts. For example, you can create an action group for critical alerts that sends an email to the on-call engineer, creates a ticket in your service management system and triggers an automation runbook to perform a specific action.
Test your alerts: Before you start using your alerts in production, it’s a good idea to test them to make sure that they are configured correctly. You can do this by manually triggering the alert and verifying that the correct actions are taken. For example, you can test your alert by temporarily setting the threshold to a lower value and then verifying that the alert is triggered and the correct action is taken.
Monitor your alerts: Once your alerts are set up, it’s important to keep an eye on them to make sure that they are working as expected. You can do this by monitoring the alert history in the Azure portal, which shows you a record of all alerts that have been triggered and the actions that were taken in response. This will help you to identify any potential issues with your alerts and make any necessary adjustments.

By following these best practices, you can ensure that your Azure Monitor alerts are configured correctly and that they will help you quickly identify and resolve any issues with your Azure resources. By being specific, using alert suppression, action groups, testing the alerts and monitoring them you can make the most out of Azure Monitor alerts and have a more reliable monitoring system.

Note: There are some great example of how to create alerts using JSON templates available here.

Uncovering Anomalies in Time-series Data with Kusto Query Language (KQL)

1 Reply

Anomaly detection is a crucial task in monitoring the performance of various systems. In this blog post, we will discuss how to use Kusto Query Language (KQL) to detect anomalies in CPU performance data.

Spikes

One of the most common types of anomalies is spikes in the data. Spikes occur when the data deviates significantly from its normal behavior. To detect spikes in CPU usage over time, we can use the following KQL query:

let window = 24h;
Perf
| where ObjectName == "Processor" and CounterName == "% Processor Time"
| where TimeGenerated > ago(window)
| summarize avg(CounterValue),stdev(CounterValue) by bin(TimeGenerated, 2h), Computer
| where (avg_CounterValue - avg_CounterValue) > 3 * stdev_CounterValue

This query first filters the data to include only CPU usage data and only the data that is within the last 24 hours. It then groups the data by time window and computer, calculates the average and standard deviation of the data, and finally filters out any data points that are more than 3 standard deviations away from the average.

Outliers

Another type of anomaly is outliers. Outliers are data points that are significantly different from the rest of the data. To detect outliers in CPU usage across different machines, we can use the following KQL query:

Perf
| where ObjectName == "Processor" and CounterName == "% Processor Time"
| summarize percentile(CounterValue,75) by Computer
| where percentile_CounterValue_75 > 50

This query filters the data to include only CPU usage data, calculates the 75th percentile of the data for each computer, then filters the results and only show the computers that have 75th percentile values higher than 50.

Changes over time

Finally, another type of anomaly is changes in the data over time. To detect changes in CPU usage over time, we can use the following KQL query:

let window = 7d;
Perf
| where ObjectName == "Processor" and CounterName == "% Processor Time"
| where TimeGenerated > ago(window)
| summarize avg(CounterValue) by Computer, TimeGenerated = startofday(TimeGenerated)
| join (
    Perf
    | where ObjectName == "Processor" and CounterName == "% Processor Time"
    | where TimeGenerated > ago(window)
    | summarize arg_min(TimeGenerated, CounterValue) by Computer, TimeGenerated = startofday(TimeGenerated)
    | where TimeGenerated < TimeGenerated
    | project Computer, TimeGenerated, CounterValue
) on Computer, TimeGenerated
| extend diff = avg_CounterValue - CounterValue
| where abs(diff) > 10

This query filters the data to include only CPU usage data and only the data that is within the last 7 days. It then groups the data by day and computer, calculates the average of the data, and finds the difference between consecutive days’ averages. The query finally filters out any data points where the difference is greater than 10.

Summary

In this blog post, we have discussed how to use KQL to detect different types of anomalies in CPU performance data. These queries can be customized and adjusted to fit the specific needs of your system and can be a valuable tool in monitoring and maintaining the performance of your systems. Anomaly detection can be complex but is also very powerful.

Kusto Detective Agency: Challenge 5 – Big heist

24 Replies

Challenges

Challenge 1: Here
Challenge 2: Here
Challenge 3: Here
Challenge 4: Here
Challenge 5: This article

The ADX team upped their game once again. Time for a proper forensic investigation, track down the baddies, find clues and decipher their meaning all while racing against the clock. Can you come up with the date and location of the heist in time to stop them?

General advice

This challenge requires a bit of creative thinking, even with the hints there are multiple paths to go down and not all of them are going to lead to the right outcome. the key to this one, keep it simple and logical.

Challenge 5: Big heist

This challenge also has multiple parts, first we need to identify four chatroom users from over three million records and then we need to “hack” their IPs to get more clues.

Query Hint Part 1

Trying to identify the right user behaviors here is super tricky, I got tripped up here by adding a level of complexity that was unnecessary. At its simplest we would have to find a room where only 4 people joined and no one else. Some KQL commands that will be useful here are tostring, split, extend, row_cumsum

Big heist challenge text - Part 1

Hello. It’s going to happen soon: a big heist. You can stop it if you are quick enough. Find the exact place and time it’s going to happen.
Do it right, and you will be rewarded, do it wrong, and you will miss your chance.

Here are some pieces of the information:
The heist team has 4 members. They are very careful, hide well with minimal interaction with the external world. Yet, they use public chat-server for their syncs. The data below was captured from the chat-server: it doesn’t include messages, but still it may be useful. See what you can do to find the IPs the gang uses to communicate.
Once you have their IPs, use my small utility to sneak into their machine’s and find more hints:
https://sneakinto.z13.web.core.windows.net/<ip>

Cheers
El Puente

PS:
Feeling uncomfortable and wondering about an elephant in the room: why would I help you?
Nothing escapes you, ha?
Let’s put it this way: we live in a circus full of competition. I can use some of your help, and nothing breaks if you use mine… You see, everything is about symbiosis.
Anyway, what do you have to lose? Look on an illustrated past, fast forward N days and realize the future is here.

Query challenge 5 - Part 1

Alright we’ve got some IPs, so time to “hack”, using the provided tool you’ll gather a set of clues from each of the gang members, there are a few key things you need to find, these are an email, some pictures, a cypher tool, an article and a pdf copy of it and of course a video from the nefarious professor Smoke.

From here on out it’s all investigative skills, you now have everything you need to find the date and location of the heist and save that datacenter!

Final hint

In order to decrypt the secret message, you’re going to need a special key, the format looks familar right? Spot on you’ll need the answer from challenge 4!

Congratulations Detective!

If you’ve found this blog series useful, please let me know via LinkedIn or drop a comment below. These challenges have been super fun and I for one am looking forward to season 2!