What is Azure Purview?

For some time, Microsoft has been working on the internal project under the codename “Babylon” for restructuring the data catalog service on Azure. Now, it is officially revealed that the data product is “Azure Purview”.

“Azure Purview is a unified data governance platform which automates the discovery of data, cataloging of data, mapping of data, lineage tracking — with the intention of giving our customers a very good understanding of the breadth of the data estate that exists to begin with, and also to ensure that all these regulations that are there for compliance, like GDPR, CCPA, etc, are managed…

Co-authors: Gananda Hayardisi, Wipada Chanthaweethip

Last year, it was obvious from the Microsoft BI official announcement that Power BI Premium is a superset of Azure Analysis Services. You may check out my old post below.

Unblocking Power BI — Power BI Premium as a Superset of Analysis Services (Ep. 5) | by Korkrid Akepanidtaworn (Kyle) | Dev Dream Team | Medium

The concept of data mart vs. data warehouse has been one of the biggest hurdles to my customers. The difference comes in three aspects:

1. Data Size — a data mart is typically less than 100 GB; a data warehouse…

In the early days of data repository, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a commonly known system used for reporting and data analysis, and is considered a core component of business intelligence. Extract, transform, load (ETL) and extract, load, transform (E-LT) are the two main approaches used to build a data warehouse system. DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place. It may sound remarkably similar to the definition of a data lake. In particular, data…

Using Power BI to Automate Reporting Lifecycle (BIOps)

Previous Episode(s)

• Unblocking Power BI — Can Power BI Report Server (On-Premise) Export Data in CSV and Excel? (Ep. 1)
• Unblocking Power BI — Dashboard in a Day (Ep. 2)
• Unblocking Power BI — How to Acquire Power BI Desktop On-Premise vs. Cloud? (Ep. 3)
• Unblocking Power BI — Let’s be Clear about Power BI Product Portfolio! (Ep. 4)
• Unblocking Power BI — Power BI Premium as a Superset of Analysis Services (Ep. 5)

Power BI Service has a concept of workspaces where you create collections of dashboards, reports, and paginated reports and collaborate with colleagues with whomever inside the…

Tools and Technologies

• Azure DevOps — Services for teams to share code, track work, and ship software
• Azure Machine Learning — Bring AI to everyone with an end-to-end, scalable, trusted platform with experimentation and model management. Easily build, deploy, and manage predictive analytics solutions.

Here’s a full directory of Azure Cloud Services

Previous Episodes:

• MLOps on Azure End-to-End (E2E) Playbook (Ep. 1)
• MLOps on Azure End-to-End (E2E) Playbook (Ep. 2)
• MLOps on Azure End-to-End (E2E) Playbook (Ep. 3)

Machine learning projects usually follow the workflow where data experts split the data into training and testing. Both of which go through…

Prelude

The new era of cloud computing begins…I discovered RStudio Cloud (currently in beta release at the time I am writing) suitable for professionals, hobbyists, trainers, teachers and students to do, share, teach and learn data science using R. For those who don’t know, RStudio is a full-fledged IDE for R Programming. Here’s an exact view you are going to be using as the Desktop version designed to solve some pain points such as:

• Install R
• Install RStudio
• Install packages and dependencies
• Load packages
• Run and re-load the codes

Now, with the new cloud experience, I can simply log in and…

Previous Episode(s)

• Unblocking Power BI — Can Power BI Report Server (On-Premise) Export Data in CSV and Excel? (Ep. 1)
• Unblocking Power BI — Dashboard in a Day (Ep. 2)
• Unblocking Power BI — How to Acquire Power BI Desktop On-Premise vs. Cloud? (Ep. 3)
• Unblocking Power BI — Let’s be Clear about Power BI Product Portfolio! (Ep. 4)

Power BI Premium is a capacity-based subscription license, meaning that increments of capacity in the form of virtual CPU cores and memory are purchased. Typically, this license is chosen for large-scale BI deployment project serving a large number of users (both…

Last Updated: May, 21, 2020

Pandas Dataframes

• Pandas Dynamically Create a Dataframe
• Get a filtered list of files in a directory
• Read different .csv files into different dataframes with a loop with Python Pandas
• Import multiple csv files into pandas and concatenate into one DataFrame (glob function)

Filters

• How to filter rows containing a string pattern from a Pandas dataframe
• Ignoring NaNs with str.contains

Regular Expression

• Remove all special characters, punctuation and spaces from string
• Python RegEx W3Schools

For-Loop Operations

• Python: Add value of iteration variable to variable name within for loop

Export Dataframes

• How to save CSV files with…