Microsoft Azure Data Lake Architecture

Big data solutions typically involve a large amount of non relational data such as key value data json documents or time series data.

Microsoft azure data lake architecture.

Authenticate data using azure active directory azure ad and role based access control rbac. Azure data lake works with existing it investments for identity management and security for simplified data management and governance. Azure includes many services that can be used in a big data architecture. Managed services including azure data lake store azure data lake analytics azure synapse analytics azure stream analytics azure event hub azure iot hub and azure data factory.

There isn t a single measure or tool to avoid all possible problems with your data lake while good practices will protect your environment from this kind of disaster. Data lake stores are optimized for scaling to terabytes and petabytes of data. Architecture optimize administration of sql server instances in on premises and multi cloud environments by leveraging azure arc optimize management maintenance and monitoring of sql server with azure arc enabled sql server and azure arc enabled data services in on premises and multi cloud environments. And help protect data with security features like encryption at rest and advanced threat protection.

This scalable cloud data lake offers a single storage structure for multiple analytic projects of different sizes. It also integrates seamlessly with operational stores and data warehouses so you can extend current data applications. It might also support self service bi using the modeling and visualization technologies in microsoft power bi or microsoft excel. Microsoft azure data lake architecture is helping data scientists engineers and analysts by solving much of their big data dilemma.

The data typically comes from multiple heterogeneous sources and may be structured semi structured or unstructured. The data may be processed in batch or in real time. Azure stream analyticsreal time analytics on fast moving streams of data from applications and devices machine learningbuild train and deploy models from the cloud to the edge azure analysis servicesenterprise grade analytics engine as a service azure data lake storagemassively scalable secure data lake functionality built on azure blob storage. They fall roughly into two categories.

To empower users to analyze the data the architecture may include a data modeling layer such as a multidimensional olap cube or tabular data model in azure analysis services. Data governance tools like azure data catalog can help to register and access the data assets but it is not enough to avoid a data swamp. Our online certification helps you learn azure data lake from basic to advanced levels.