IRiS Walkthrough Using Microsoft Fabric

01. About this walkthrough

This walkthrough uses a fixed reference data set to take you from an empty Fabric workspace to a generated IRiS model. You stage the data in Fabric, model three source tables in IRiS, and push the generated metadata and code to your repository. Because the data is identical in every environment, you can check each result against the expected outcomes in this guide.

It assumes working knowledge of Microsoft Fabric. Allow 60–90 minutes.

02. The scenario

The reference data set describes an Australian distributor of hardware and components. Marketing runs a CRM and finance runs an ERP, and each holds its own version of the same customers. The ERP also produces a wide order-line extract.

The walkthrough shows IRiS doing three things:

1. Integrating customers across systems. The two customer tables share 1,000 keys under different column names. IRiS brings both onto one Customer entity, with a set of properties per source.
2. Resolving a wide table. Order and product details repeat on every order line. IRiS moves them to the right grain.
3. Growing a shared model. The order-line table reuses the Customer entity created by the first two sessions.
   
   

03. Before you start

• A Fabric workspace on an active capacity (F2 or above, or a trial), with Contributor access.
• Access to IRiS, with Settings configured or rights to configure them.
• A Git repository IRiS can push to, and a model provider for IRiS.
• The setup notebook 01_setup_bronze_layer.py.

04. Part A: Set up the data in Fabric

1. Create three lakehouses with schemas enabled: lh_bronze_crm, lh_bronze_erp and lh_silver. Schemas are mandatory. Every object is addressed as schema.table, and a lakehouse without schemas will not resolve them.
2. Import the notebook. Workspace toolbar → Import → Notebook → 01_setup_bronze_layer.py.
3. Attach all three lakehouses to the notebook, with lh_bronze_crm as the default. Attachments do not carry over on import, and the notebook fails if any lakehouse is missing.
4. Run all. The notebook creates the landing schema in both bronze lakehouses, creates the landing, load, stage, vault, sorv and im schemas in lh_silver, and seeds the three tables. Re-running it resets the data to the same clean state.

Check the printed output:

5. Create shortcuts in silver. IRiS reads from the lh_silver SQL analytics endpoint. In lh_silver, select the landing schema → New shortcut → Microsoft OneLake → pass-through identity, and create a table shortcut to each bronze table: customer_crm, customer_erp and order_lines_erp. Use shortcuts rather than Spark views, because views do not appear on the endpoint.
6. Verify on the SQL endpoint.

This should return 1,000:

SELECT COUNT(*) AS shared_customers
FROM landing.customer_crm c
JOIN landing.customer_erp e ON c.crm_customer_id = e.customer_number;

05. Part B: Configure IRiS

Set these once in the Settings tab:

06. Part C: Model the three tables

Run one IRiS session per table, in this order: customer_erp, customer_crm, order_lines_erp.

The customer tables establish the Customer entity, and the order lines build on it.

For all three tables, the answers to the source questions are the same: not CDC (full extract), raw source data, and no extract date, so let IRiS use the load date. Use placeholder source owners (for example “ERP team” and “CRM team”).

Use the same collision code for every session. The CRM and ERP share identifiers for the same customers. If each source gets its own collision code, IRiS treats them as different customers and they will not integrate.

Session 1: customer_erp

Session 2: customer_crm

Open the Model view and select Customer. It should show two contributing sources, CRM and ERP.

Session 3: order_lines_erp

Expected model:

Check that IRiS has moved the order and product details off the line. If not, select Request revision or tell IRiS in the chat, for example: “order_date, order_status and order_amount describe the order, not the line.”

07. Part D: Deploy and run in Fabric

At the end of each session, IRiS can push a package of Fabric notebooks to your repository. This reference guide shows you how to run the package by hand in the workspace, using four wrapper notebooks for each source table:

You only run the wrappers. Each one calls the other generated notebooks in the correct order.

Step 1 - Bring the notebooks into Fabric

1. In your workspace, open Workspace settings → Git integration.
2. Connect to the repository, branch and folder IRiS pushed to.
3. Select Source control → Update all.
4. Make sure the notebooks use lh_silver as their default lakehouse. If a notebook reports a missing table or schema, check its default lakehouse first.

Step 2 - Deploy the tables

Run each Deploy wrapper:

• Deploy_customer_erp
• Deploy_customer_crm
• Deploy_order_lines_erp

Shared tables such as the Customer hub are created by the first wrapper and left unchanged by the others.

Step 3 - Create the ghost records

Run each ghost record wrapper:

• Create_ghost_record_customer_erp
• Create_ghost_record_customer_crm
• Create_ghost_record_order_lines_erp

Ghost records are placeholder rows with keys starting ~. They let joins across the model return a row even when there's no matching data. Running these wrappers again is safe, because existing ghost records are not duplicated.

Step 4 - Load the data

Run each Ingest wrapper, in the same order:

• Ingest_customer_erp_1
• Ingest_customer_crm_1
• Ingest_order_lines_erp_1

Each wrapper loads its source from lh_silver.landing, stages it with hash keys, and loads the hubs, links and satellites. Running them again is safe, because unchanged records are not loaded again.

Step 5 - Check the results

Run these checks on the lh_silver SQL analytics endpoint. Exclude the ghost records, which have keys starting with ~.

-- One customer, two systems
SELECT COUNT(*) AS customers
FROM vault.h_customer
WHERE bk_customer NOT LIKE '~%';
-- Expect 1,400. Each landing table holds 1,200 rows.

This is the key result: two systems holding 1,200 customers each become one Customer entity with 1,400 real customers.

Step 6 - Clean up (optional)

The cleanup wrappers remove the generated notebooks from the workspace. They do not drop tables or delete data.

• Cleanup_ddl_<table> removes the notebooks that create the tables. You can run it safely once the tables are deployed.
• Cleanup_proc_<table> removes the load and ghost record notebooks. After you run it, you can't reload that table until you sync the package from the repository again.

Run the cleanup wrappers when you've finished with a package, or before you sync a regenerated one.

Resetting

To reload from clean source data, re-run the setup notebook from Part A, then run the Ingest wrappers again.

08. Troubleshooting and notes

Flags such as marketing_consent and backorder_flag are stored as 0/1 integers, because IRiS does not support the native boolean type. The same reference data set will be available for Snowflake and Databricks; only Part A changes.

09. Open items for review

Delete before publishing.

• order_lines_erp has no line number, and 8 order + SKU pairs repeat, so the order line may not be unique on its keys. Consider adding line_number to the seed.
• Write the deployment steps in Part D, and a silver/registry reset procedure.
• Confirm how IRiS asks the collision code question, and that the expected models match a real run.
• Capture screenshots from a clean run.