Wednesday, February 1, 2023
No menu items!
HomeCloud ComputingIntroducing Cloud Logging - Log Analytics, powered by BigQuery

Introducing Cloud Logging – Log Analytics, powered by BigQuery

Logging is a critical part of the software development lifecycle allowing developers to debug their apps, DevOps/SRE teams to troubleshoot issues, and security admins to analyze access. Cloud Logging provides a powerful pipeline to reliably ingest logs at scale and quickly find your logs. Today, we’re pleased to announce Log Analytics, a new set of features in Cloud Logging available in Preview, powered by BigQuery that allows you to gain even more insights and value from your logs. 

Introducing Log Analytics

Log Analytics brings entirely new capabilities to search, aggregate, or transform logs at query time directly into Cloud Logging with a new user experience that’s optimized for analyzing logs data through the power of BigQuery . BigQuery is a cost-effective, serverless, multi cloud data warehouse to power your data-driven innovation. 

With Log Analytics, you can now harness SQL (see figure 1) and the capabilities of BigQuery to analyze their logs.  Now, Cloud Logging offers the functionality you had in the past and analytical capabilities through Log Analytics 

Secure, compliant and scalable logging ingestion pipeline through Logs Router

Managed logging-as-a-service solution with a specialized user interface for log analysis. 

Support for centralized logging across Google Cloud, other clouds, and on prem

Automated insights and suggestions such as Error Reporting

Log based metrics and alerts for real-time aggregation, visualization and alerting of logs

Flexible pay as you go pricing

NEW – Powerful BigQuery engine and SQL option for ad hoc log processing

NEW – Automatic read only access to all log analytics logs in BigQuery

NEW – Rich visualization of log data (Figure-2, in private preview)

Why is Log Analytics powerful?

Log Analytics leverages the power of BigQuery to enable Cloud Logging users to perform Analytics on Log data. 

Centralized logging –  By collecting and centrally storing the log data in a dedicated Log Bucket, it allows multiple stakeholders to manipulate their data from the same datasource. You don’t need to make duplicate copies of the data.

Reduced cost and complexity – Log Analytics allows reuse of data across the organization, effectively saving cost and reducing complexities. 

Ad hoc log analysis – It allows for ad-hoc query-time log analysis without requiring complex pre-processing. 

Scalable platform – Log Analytics can scale for observability using the serverless BQplatform and perform aggregation at petabyte scale efficiently

Log Analytics is designed for multiple users in an organization and aims to break down silos. Here are the top categories we hear from our users: 

Developers & DevOps use it for Infrastructure and Application troubleshooting

Security Teams use it for Audit Log Analysis. 

Networking professionals use it to perform network log analysis

Business Operations Teams can potentially manipulate the data, create KPIs and in the future create dashboards.  

Pricing

Log Analytics is included in the standard Cloud Logging pricing. Queries submitted through the Log Analytics user interface do not incur any additional cost. Enabling analysis in BigQuery is optional and, if enabled, queries submitted against the BigQuery linked data set including Data Studio, Looker and via BigQuery API, incur the standard BigQuery query cost.

Get started

Visit the Log Analytics page in the Cloud Console and upgrade an existing Log Bucket or create a new Log Bucket. Check out our sample queries to get you started. Charting in Log Analytics is available now as a Private Preview (sign-up here). In the next blog post, we will talk about how and when users should leverage Log Analytics, how you will get started with Log Analytics and dive into a few common use cases.  You can join the discussion in our Cloud Operations page on the Google Cloud Community site.

Related Article

Announcing new simple query options in Cloud Logging

The faster you can find logs, the faster you can resolve issues. Today, we’re pleased to announce a simpler way to find logs in Logs Expl…

Read Article

Cloud BlogRead More

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments