Taming the Data for Better BI
The secret to successful business intelligence is data governance and a metadata repository that connects data to standardized definitions across the campus.
Facing tight budgets and long to-do lists, some CIOs decide that creating an infrastructure for managing and categorizing data is a low priority. But institutions such as Notre Dame (IN) and the University of Washington are recognizing that without a metadata repository and a strong data governance effort, business intelligence efforts are likely to fizzle out.
"We had a data warehousing initiative seven years ago that failed," admitted Chris Frederick, Notre Dame's business intelligence manager. A post-mortem placed the blame on data governance shortcomings. "We knew we had to get everyone on the same page," he said. So when the institution kicked off its new business intelligence effort, called dataND, data governance was positioned front and center. Notre Dame hired a campus data steward and put him in charge of creating a repository and getting business users to agree on terms.
Helping Users Navigate the Data
In 2015, the University of Washington began work on its own repository called the Knowledge Navigator, which is designed to give context to the enterprise data warehouse and allow business users to see relationships between concepts, terms, tables, columns and reports. "Someone who is exploring a business question such as how many women graduated with STEM degrees last year can find agreed-upon definitions of terms like STEM and then navigate to the database," explained Matt Portwood, a UW metadata analyst.
Most such repositories are designed for metadata management by data architects, noted Pieter Visser, a UW solutions architect. "They are not created for the end-user at all," he said. In contrast, Knowledge Navigator was intended to be a tool for everybody. Visser described it as being like Google for your metadata: "We try to make it as easy as possible to find how everything is related to everything else. You can start with your business terms and go all the way to the Tableau visualization or web service, and we give you the context right away."
In their metadata repository work, both UW and Notre Dame use graph database technology from Neo4j to represent entities and their relationships. Visser explained that within the metadata world, everything is related to everything else. "A resource in a web service or a label on a report can relate to a business term or a concept," he said. "In a graph database you can easily connect any node to another node. Trying to do it in a relational database is almost impossible."
Dealing with metadata can seem like an esoteric exercise that involves semantics and data science. To make it more relevant to business leaders, the UW team tied the Knowledge Navigator project to a human resources system modernization. "We are going from a mainframe system built in 1982 to a cloud-based system from Workday," Visser said. The team decided to use Knowledge Navigator to help users understand where to find data in the new system. "We used that project as a way to leverage metadata," he explained. "There are finance and student information system modernizations coming up too. We can reuse the tool for that."
In the course of the upgrade, Portwood recalled, he met with a project manager — someone who definitely wouldn't describe herself a data person. "She pulled up Knowledge Navigator and went to one of our diagrams intended to help people have a sense of how Workday handles data," he said. From this diagram, she was able to understand the context, and click to get to a business glossary where everything is defined. She shared the information in a meeting with other project managers in different functional groups. "That is what we want — to serve different types of users, whether they are business users or report consumers or data analysts," he added. "We are working on refining the tool by better understanding our users and then meeting their specific needs."
Agreeing on Definitions and Policies
A lot of front-end data governance work is required before a metadata tool can be of value. "While we have been building out [Knowledge Navigator], we have a metadata manager who creates a grassroots governance framework with people in areas such as student academics, finance and research," Portwood said. For instance, a term such as "country of origin" means something specific in a student context but something else in a research context. "These are the kinds of nuances that the metadata manager is working through at a personal level," he noted. "There is no technology fix for that."
Portwood said there has been widespread demand for common data definitions at UW. "Our registrar's office and the graduate schools were clamoring for it," he said. "That was how we were able to engage with them, create this framework and get some quick wins."
It's a similar situation at Notre Dame, where Frederick described the lack of agreed-upon data definitions as akin to the United States before the states had a common currency. "That didn't really work well," he said. "If we can all work in the same currency, our data economy will improve. That is the underlying principle."
Beyond data definitions, metadata also allows Notre Dame to manage data security in the data warehouse. Is the data restricted and if so, how? "We can build a view from the metadata that enforces the security of the data warehouse, and which things you can access based on your role," he said.
In its effort to build a more open and transparent culture around access to data, Notre Dame has created an acceptable use policy for data. It includes guidelines for how you can decide data is restricted and what happens if employees fail to uphold those restrictions. "It is not enough to decide what the data is called; then you have to decide who should have access to it," Frederick asserted. "There are some other pillars we haven't tackled yet, such as retention and archiving. But we are chipping away at it in an effort to make data more broadly available on campus."
At dataND steering committee meetings, the university's vice presidents sing the praises of the data governance effort. "It is like shining a flashlight into the dark corners of your university," Frederick said. "Data is an asset like money. Have we been managing it well and making wise investments with our data? Do we have good processes that ensure that the data that goes in gets a good return when we pull it out? In a lot of cases we do not. That is inhibiting our ability to get value from it. But what we are finding is that without governance, we would not even know that is the issue."
Frederick said he wouldn't want to work at a university that didn't take governance and metadata seriously. "Without this, there is only so far you can go with business intelligence."