Data Mesh: Practical Guidelines for Implementation

14-15 maj, 2024

Practical Guidelines for Implementing a Data Mesh

Most companies today are storing data and running applications in a hybrid multi-cloud environment. Analytical systems tend to be centralized and siloed like data warehouses and data marts for BI, Hadoop or cloud storage data lakes for data science and stand-alone streaming analytical systems for real-time analysis. These centralized systems rely on data engineers and data scientists working within each silo to ingest data from many different sources, clean and integrate it for use in a specific analytical system or machine learning models. There are many issues with this centralized, siloed approach including multiple tools to prepare and integrate data, reinvention of data integration pipelines in each silo and centralized data engineering with a poor understanding of source data unable to keep pace with business demands for new data. Also, master data is not well managed.

To address these issues, a new approach has emerged attempting to accelerate the creation of data for use in multiple analytical workloads. That approach is Data Mesh. Data Mesh is a decentralized business domain-oriented approach to data ownership and data engineering to create a mesh of reusable data products that can be created once and shared across multiple analytical systems and workloads. A Data Mesh can be implemented in a number of ways. These include using one or more cloud storage accounts on cloud storage, on an organized data lake, on a Lakehouse, on a data cloud, using Kafka or using data virtualization. Data products can then be consumed in other pipelines for use in streaming analytics, Data Warehouses or Lakehouse Gold Tables, for use in business intelligence, data science and other analytical workloads.

This 2-day course looks at:

  • Data Mesh in detail and examines its strengths, and weaknesses
  • The strengths and weaknesses of Data Mesh implementation options
  • Which architecture is best to implement a Data Mesh
  • How to co-ordinate multiple domain-oriented teams
  • How to use of common data infrastructure software like Data Fabric to create high-quality, compliant, reusable, data products in a Data Mesh
  • How to use a data marketplace to share data products
  • The objective to shorten the time to value while also ensuring that data is correctly governed and engineered in a decentralized environment
  • Organizational implications of a Data Mesh
  • How to create sharable data products for master data management and for use in multi-dimensional analysis on a data warehouse, data science, graph analysis and real-time streaming analytics to drive business value
  • Technologies like data catalogs, Data Fabric for collaborative development of data integration pipelines to create data products, DataOps to speed up the process, data orchestration automation, data marketplaces and data governance platforms

Why attend

You will learn about:

  • Strengths and weaknesses of centralized data architectures used in analytics
  • The problems caused in existing analytical systems by a hybrid, multi-cloud data landscape
  • What is a Data Mesh and how does it differ from a Data Lake and a Data Lakehouse?
  • What benefits does a Data Mesh offer and what are the implementation options?
  • What are the principles, requirements, and challenges of implementing these approaches?
  • How to organize to create data products in a decentralized environment so you avoid chaos
  • The critical importance of a data catalog in understanding what data is available
  • How business glossaries can help ensure data products are understood and semantically linked
  • An operating model for effective federated data governance
  • What software is required to build, operate and govern a Data Mesh of data products for use in a Data Lake, a Data Lakehouse or Data Warehouse?
  • What is Data Fabric software, how does it integrate with data catalogs and connect to data in your data estate
  • An implementation methodology to produce ready-made, trusted, reusable data products
  • Collaborative domain-oriented development of modular and distributed DataOps pipelines to create data products
  • How a data catalog and automation software can be used to generate DataOps pipelines
  • Managing data quality, privacy, access security, versioning and the lifecycle of data products
  • Publishing semantically linked data products in a data marketplace for others to consume and use
  • Consuming data products in an MDM system
  • Consuming and assembling data products in multiple analytical systems like data warehouses, lakehouses and graph databases to shorten time to value

Who should attend

This course is intended for business data analysts, data architects, chief data officers, master data management professionals, data scientists, ETL developers and data governance professionals.

Prerequisites

This course assumes you understand basic data management principles and data architecture plus a reasonable understanding of data cleansing, data integration, data catalogs, data lakes and data governance.

Instructor

Mike Ferguson is the Managing Director of Intelligent Business Strategies Limited. As an independent IT industry analyst and consultant, he specializes in BI/Analytics and data management. With over 40 years of IT experience, Mike has consulted for dozens of companies on BI/analytics, data strategy, technology selection, data architecture and data management.

Mike is also conference chairman of Big Data LDN, the fastest-growing data and analytics conference in Europe and a member of the EDM Council CDMC Executive Advisory Board. He has spoken at events all over the world and written numerous articles.

Formerly he was a principal and co-founder of Codd and Date Europe Limited – the inventors of the Relational Model, a Chief Architect at Teradata on the Teradata DBMS.

He teaches popular master classes in Data Warehouse Modernization, Big Data Architecture & Technology, How to Govern Data Across a Distributed Data Landscape, Practical Guidelines for Implementing a Data Mesh (Data Catalog, Data Fabric, Data Products, Data Marketplace), Real-Time Analytics, Embedded Analytics, Intelligent Apps & AI Automation, Migrating your Data Warehouse to the Cloud, Modern Data Architecture and Data Virtualisation & the Logical Data Warehouse.

The event: 14-15 May 2024

Instructions start at 9.00 am and end at 17.00 pm. Lunch, morning, and afternoon snacks are included. Let us know if you have any dietary restrictions or any other special needs.

Venue
The venue for this class is hosted by Avega Group.
Avega Group – The Elevate Room is in Sturegallerian, one of Stockholm’s most prestigious addresses. The entrance is located opposite restaurant Tures.

Address
Avega – The Elevate Room
Grev Turegatan 11A, 3rd floor, 114 46 Stockholm, Sweden

Format
On-site at Avega Elevate venue

Price
1.450,00 (+VAT) per person.

We offer the following discounts:

  • 10% discount for groups of 2 or more students from the same company registering at the same time.
  • 20% discount for groups of 4 or more students from the same company registering at the same time.

Cancellation policy

In case of no show, the full course amount will be charged. If you are unable to use your place, for example due to illness, it may be transferred to another person in consultation with the organizer. (Any license costs will be charged if a replacement is not found.) Cancellation must be made no later than 2 weeks before the course. If the course is canceled, the course fee will be refunded in full.

Registration and more information!

I accept that the information I provide will be saved by the recipient and possibly used to contact me in the future to send me relevant information about other events that you think might interest me. I can easily unsubscribe from these mailings if I no longer wish to receive information from Avega by emailing gdpr@avega.se

Develop with Avega!

Elevate is Avega’s forum for competence development where our employees’ interests and needs control the content. The purpose is to offer a platform where both our employees and our customers continuously develop.

Avega improves companies, society and people’s lives through digital transformation. Avega was founded in 2000 and has been a subsidiary of Tietoevry since December 2017. Avega is established in Stockholm, Gothenburg and Malmö.

 

Dela