[ad_1]
As businesses today look to get more from data, take full advantage of the cloud, and move into the age of AI, they are looking for services that process data at scale, reliably, and efficiently. Today we’re excited to announce the upcoming public preview of HDInsight on Azure Kubernetes Service (AKS), our cloud-native, open source big data service, completely rebuilt on the Azure Kubernetes Service infrastructure with two new workloads and numerous improvements across the stack. The public preview will be on October 10th. be available.
HDInsight on AKS gain performance
HDInsight on AKS includes Apache Spark, Apache Flink, and Trino workloads on an Azure Kubernetes Service infrastructure and offers deep integration with popular Azure analytics services such as Power BI, Azure Data FactoryAnd Azure Monitor, while Azure Managed Services for Prometheus and Grafana are used for monitoring. HDInsight on AKS is an end-to-end open source analytics solution that is easy to deploy and cost-effective to operate.
HDInsight on AKS helps customers leverage open source software for their analytics needs by:
- Delivering a curated set of open source analytics workloads such as Apache Spark, Apache Flink and Trino. These workloads are best-in-class open source data engineering, machine learning, streaming, and query software.
- Providing managed infrastructure, security, and monitoring so teams can spend their time building innovative applications without worrying about the other components of their stack. The teams can rely on that HDInsight helps protect your data.
- Provides the flexibility teams need to expand their capabilities by leveraging today’s rich open source ecosystem for reusable libraries and customizing applications through scripted actions.
Customers investing heavily in open source analytics can use HDInsight on AKS to reduce costs by setting up fully functional end-to-end analytics systems in minutes and leveraging pre-built integrations, built-in security, and reliable infrastructure. Our investments in performance improvements and features like autoscaling enable customers to run their analytics workloads at optimal cost. HDInsight on AKS has a very simple and consistent pricing structure per Vcore per hour, regardless of resource size or region, plus the cost of the deployed resources.
Developers love HDInsight for the flexibility it offers to extend the basic functionality of open source workloads through scripting actions and library management. HDInsight on AKS provides an intuitive portal experience for managing libraries and monitoring resources. Developers have the flexibility to use a Software Development Kit (SDK), Azure Resource Manager (ARM) templates, or the portal experience, depending on their preference.
Join us as we take a deep dive into this introduction in our coming year free webinar.
Open, managed and flexible
HDInsight on AKS covers the full range of business analytics needs spanning streaming, query processing, batch and machine learning tasks with unified visualization.
Curated open source workloads
HDInsight on AKS includes workloads selected based on their use in typical analytics scenarios, community adoption, stability, security, and ecosystem support. This ensures that customers do not have to deal with this Complexity of choice due to countless offerings with overlapping features and inconsistent interoperability.
Each of the workloads on HDInsight on AKS is best-in-class for the supported analytics scenarios:
- Apache Nimble is the open source distributed stream processing framework that enables stateful stream processing and enables real-time analysis scenarios.
- Trino is the federated query engine that is high-performance and scalable, enabling ad hoc queries across a variety of data sources, both structured and unstructured.
- Apache Spark is the trusted choice of millions of developers for their data engineering and machine learning needs.
HDInsight on AKS delivers these popular workloads with a common authentication model, common meta-store support, and pre-built integrations that make deploying analytics applications easier.
Managed service reduces complexity
HDInsight on AKS is a managed service on the Azure Kubernetes Service infrastructure. With a managed service, customers are not burdened with managing infrastructure and other software components, including operating systems, AKS infrastructure and open source software. This ensures that companies can benefit from ongoing security, feature and performance improvements without investing valuable development hours.
Containerization enables seamless deployment, scaling, and management of key architectural components. The inherent resiliency of AKS allows for automatic rescheduling of pods on newly commissioned nodes in the event of failures. This means orders can be executed with minimal disruption to Service Level Agreements (SLAs).
Customers combining multiple workloads in their data lakehouse must deal with different user experiences, resulting in a steep learning curve. HDInsight on AKS provides a unified experience for managing your lake home. Deploy, manage and monitor all workloads from a single interface. Additionally, managed services for Prometheus and Grafana allow administrators to monitor cluster health, resource usage, and performance metrics.
The auto-scaling capabilities included in HDInsight on AKS allow resources – and therefore costs – to be optimized based on usage requirements. For jobs with predictable workload patterns, teams can schedule resources to autoscale based on a predefined schedule. Proper decommissioning allows definition of wait times for jobs to be completed before resources are reduced, elegantly balancing cost and experience. Load-based autoscaling can increase and decrease resources based on usage patterns measured by compute and storage usage.
HDInsight on AKS marks a departure from traditional security mechanisms such as Kerberos. It includes OAuth 2.0 as a security framework and provides a modern and robust approach to protecting data and resources. In HDInsight for AKS authorization, access controls are based on managed identities. Customers can also bring their own virtual networks and link them during cluster setup to increase security and enable compliance with their corporate policies. The clusters are isolated with namespaces to protect data and resources within the tenant. HDInsight on AKS also enables cluster access management using Azure Resource Manager (ARM) roles.
Customers who participated in the private preview love HDInsight on AKS.
Here’s what one user had to say about their experience.
“With HDInsight on AKS, we have seamlessly transitioned from the limitations of our internal solution to a robust managed platform. This critical shift means our engineers now have the freedom to focus their expertise on core business innovation, rather than on platform management. The harmonious integration of HDInsight with other Azure products has increased our efficiency. Enhanced security strengthens the integrity and trustworthiness of our data, while scalability ensures we can grow smoothly. Essentially, HDInsight on AKS strengthens our data strategy and enables more efficient and effective business operations.”
Matheus Antunes, Data Architect, XP Inc
Azure HDInsight for AKS resources
[ad_2]
