10 Azure Log Analytics Workspace Best Practices
What is Azure Log Analytics Workspace
Azure Log Analytics Workspace is a unique environment for log data from Azure Monitor and other Azure services such as Microsoft Sentinel and Microsoft Defender for Cloud. Each workspace has its own data repository and configuration but can combine data from multiple services.
What is the need of Azure Log Analytics Workspace
A Log Analytics Workspace can be considered as the fundamental unit of management for Azure Monitor logs. All data collected by monitors can be stored and managed in one place. A Log Analytics Workspace acts as a logical storage unit where it can easily store, persist, and query data collected from various resources monitored in Azure to gain valuable insights about those resources.
How to create Log Analytics Workspace in Azure
In the Azure portal, click All Services and navigate to Monitor from the list of services. Once Monitors is selected it will display groups of resources. Select Log Analytics Workspaces from the displayed resource groups. A list of previously configured Workspaces will be displayed. Click Add to create a new Log Analytics Workspace. Provide the values to create a new Workspace.
- Subscription – Select the subscription from the list displayed.
- Resource Group – Select the Resource Group from the existing resource group list or create a new resource group.
- Instance Details – Provide a name for the Log Analytics Workspace which must be unique across all Azure Monitor subscriptions.
- Region – Select the appropriate region from the list.
- Tags – Provide the tag name & value if required.
- Pricing – For Log Analytics Workspace, only Pay-As-You-Go (Per GB 2018) plan will be available under the Pricing Tier.
After updating all the required information, click on create for creation of workspace. To check Azure Log Analytics, open the Log Analytics demo environment or select Logs from Azure Monitor menu under subscription. This sets the initial scope for Log Analytics Workspace where the query selects all the data in that workspace.
Best Practice for Azure Log Analytics Workspace
Separate workspaces for each environment:
If there are multiple workspaces, then the control over data access is granular. For example, developers can have to access production data in their own workspaces and restrict access in other workspaces.
Also, the users can use workspaces to logically separate different types of data. For example, a user may need one workspace for application logs and another for system logs. Alternatively, a user may want a workspace for each client. Multiple workspaces makes it easy to remove data that is no longer needed. Deleting a workspace permanently deletes all data in that workspace.
Using same resource group for all the workspaces in the environment:
Having multiple workspaces in the environment, it can be difficult to track which workspace is being used for what purpose. Using the same resource group for all workspaces in the environment allows a user to identify which workspaces is being used for what purposes. Also, using the same resource group for all workspaces in the environment makes it easy to delete unused workspaces. If there are multiple workspaces in different resource groups, then the user will have to delete each workspace individually. However, if they are all in the same resource group, then a user can delete the entire resource group. This will delete all workspaces in that resource group.
Keep short data retention period:
It is expensive to keep the data for longer period in the workspace. The data is stored in Azure Storage, and it will charge for the amount of storage used. The more data stored, the price per gigabyte decreases, but deleting data which is not needed is cheaper than keeping it. Also, if the data is kept for longer period, it becomes more difficult to query it. As the data ages, it becomes more difficult to query as the schema changes over time. If the data is kept for a short period of time, it’s easier and less expensive to run a query.
Use Azure Monitor to check Log Analytics performance:
Azure Monitor is a service that provides performance monitoring and log collection for all Azure resources. Log Analytics is a service within Azure Monitor that collects, analyzes, and stores data generated by resources in the Azure environment. Log Analytics is built on top of Azure Monitor, so there is an advantage of all the features and capabilities that Azure Monitor offers. This includes the ability to set alerts, build dashboards, and analyze data using a query language. By using Azure Monitor to monitor Log Analytics performance, a user can get the most out of the service and quickly identify and fix any issues that may arise.
Avoid multiple queries:
When executing multiple queries in one query, each subsequent query is executed using the results of the previous query. This can lead to inaccurate results and slow performance. It’s good practice to split the query into separate queries and use the results of each query as input for the next query. This ensures more accurate results and better performance.
Avoid using unknown characters:
Using unknown characters at the beginning of words can significantly degrade query performance. This is because the Log Analytics workspace scans the entire dataset to find all possible matches. Example – There is a field named “user_id” and want to find all records that contain user ids that start with “a”, do not use the query “user_id:a*”. Instead use the query “user_id:*a*” The asterisk (*) is the only unknown character allowed at the beginning of words in Azure Log Analytics workspaces.
Avoid distinct operator, use summarize operator:
The distinct operator returns all unique values of the specified column, while the summarize operator groups those values. To get the count of numbers of unique values in a column, using distinct operator will give the exact number. However, to group these values together, using summarize operator is more efficient.
Remove unwanted results using a where clause:
Where clause is used to specify criteria for the records returned in the query results. By default, all records are returned but sometimes it displays only those records that meet certain criteria. Where clauses are very powerful and can be used to filter almost any field in the log data. However, excluding too much data can make troubleshooting difficult, so it’s important to use it judiciously. A common mistake is to use a where clause to filter out the recent records. This is useful for tracking down issues that just happened, but it can also make it harder to keep track of things. Considering only the most recent records, there is a possibility of missing something important that happened earlier. It’s best to start with a broad query that returns all relevant records and use where clause to narrow the results as needed.
Display the Charts and Tables using Render Operator:
The render operator can be used to retrieve data from the Log Analytics workspace and render it in a visual way. This is important as it makes the data easier to understand and interpret. When using the render operator, a user can choose from different chart types, including bar charts, line charts, pie charts, and tables. It can also customize the appearance of charts and tables to make them visually presentable. Using render operator is the best practice as it makes the data from the Log Analytics workspace much easier to understand. It also makes it easier to share data with others.
Reduce the column numbers in the query using Project Operator:
A project operator limits the columns returned by a query. This reduces the amount of data transferred from the Log Analytics Workspace to the client. This can have a significant performance impact, especially for large workspaces. Project Operators can also reduce the cost of storing data in the workspaces. It can reduce the amount of memory used by limiting the number of columns stored. Finally, project operators help improve query usability. By returning only the columns that are required, the users can better understand the results of the query.
Conclusion
Azure Log Analytics is used to run log queries for data in Azure Monitor logs. It is a reliable, easy-to-use, and secure system for collecting and analyzing performance information.
