Getting started with Dewi¶

Note: before you start using dewi, right now only SQL Server is supported and for Azure SQL server you need atleast the S3 tier (because of column store indices)

Intro¶

First things first. Dewi is a command line python application to generate datawarehouse projects based on Yaml configuration files. This should help you implement data flows and the necessary SQL procedures. Export data governance documents (WIP) and visualize datalineage (WIP).

Installation¶

Dewi is hosted on our private python package registry so to install it you need add the registry in your command:

pip install -i https://pkgs.dev.azure.com/AllYourBI/_packaging/AllYourBI/pypi/simple dewi

Be aware: There is also a package called 'dewi' on the official PyPI, which is not the same as our dewi package. So if you get strange warnings check which version you got and delete the dewi package before trying a reinstall with the above command

This will most likely prompt you for a Username and a Password. The Username should be the username of Azure DevOps. The Password is not your usual password but the secret value of a Personal Access Token. To create a PAT check this link: Create a Personal Acces Token (PAT) Be sure to add the persmission to 'Read' packages/feeds

Looking in indexes: https://pkgs.dev.azure.com/AllYourBI/_packaging/AllYourBI/pypi/simple
User for pkgs.dev.azure.com: Jeffrey.Slort
Password:

If you're using Pipenv to manage your virtual environment then you should add the following to your Pipfile

[[source]]
url = "https://{Username}:{Password}@pkgs.dev.azure.com/AllYourBI/_packaging/AllYourBI/pypi/simple"
verify_ssl = true
name = "artifacts"

Where {Username} and {Password} should be replaced with your actual username and PAT obviously

Usage¶

When dewi is installed it's really easy to start. First open your SQL project in Azure Data Studio or another IDE/editor. Open your terminal in this directory and run:

dewi build

This will read the config files in the 'dewi' subdirectory, validate the configs, build the datamodel and generate all necessary code

To see what other commands are available run:

dewi --help

To output more information add -v or --verbose to your command.