I have worked for 9 months as Delivery Manager Denodo at Forem, and developed its virtualized data architecture and governance.
Data virtualization offers:
- A way of reducing the growth of data volume (Microsoft does’nt like it)
- A way of controling the quality of your data (consultants don’t like it)
- A way of having business and IT to better collaborate (some managers don’t like it)
As in any transformation process, enhancement empowers some and dispowers others. A lot of change management is then needed to help the organization’s digestion.
A brief description of Denodo
Denodo basics are quite easy: queries are based on SQL, and the interface is graphical. You can even drag and drop items to create simple queries, that you then combine into complex ones. I gave 2 hours-SQL courses to business users, and they quickly could create their own queries.
Denodo’s transparency tools allow the vizualisation of data lineage and a complete overview of how the queries were automatically optimized. It’s easy, it’s comfy, it’s trustable.
DB data architecture vs virtualized data architecture
Denodo connects to any kind of data sources, and gathers data (see the above scheme).
This virtualized data is flat: it has no connections and no conditions. Only tables remain.
- PRO: you can do whatever you want with it
- CON: you can do whatever you want with it
You then access to a blank canvas where you can organize your data as needed, in virtual domains and subfolders. Some choose to name their virtual domains after projects, others after physical departments.
Denodo also offers a fully graphical interface with Denodo Data catalog, which allows data filtering. I worked with the Data management team to define categories and tags strategy.
Virtualized data governance
Virtualized data sells less Microsoft licenses and less IT consultants hours, it also allows a direct collaboration between business teams and IT teams : it needs a strong top management sponsorship to be able to fully develop .
At Forem, I worked with IT managers to organize an asynchronous virtualized data governance:
- I funded a Data virtualization community, using MS Teams and IRL meetings
- together with biweekly distant meetings with virtual domains managers
- and trainings in Denodo, in Denodo SQL, in virtual domain management
- I worked with the Data manager and the BI team to set up handy dashboards for the IT managers to overview how Denodo was used by the teams.
What I’ve learned
Data virtualization is a very virtuous way of decreasing the impact of IT on the environment, but it seems many organizations are not ready yet to adopt it. I am looking forward to Microsoft’s next moves: Microsoft Fabric is already a step into this direction.