At last week’s CDOIQ Symposium in Cambridge, MA, Tom Redmond of Data Quality Solutions and Tom Davenport, who has a ubiquitous presence in the data field, jointly presented a session called “The Rise of Tweener Roles in Data Science.” The main takeaway was that – data people who bridge gaps between lines of business and data science teams – will become increasingly important as organizations recognize that data must be front-and-center of every function to gain an advantage.
The Problem – Understanding the Gaps Between Lines of Business and Data Scientists
The problem has been that data science could be faster and can be painful and expensive. Previously it has been “bolted on” to organizations as Data Science Centers of Excellence rather than built into lines of business. While data scientists aim to disrupt, business leaders want control, stability, and no surprises.
If companies are committed to using data to transform business, both speakers agree that data science needs to become mainstream. Business leaders and data scientists compete for resources for people and executive attention. The data may need to be completed, updated, or worse – executive teams are not aligned. Resources are limited, and business leaders need to see results and do not want to disrupt their strategy.
“Purple People” – Tweeners Closing the Gaps Between Line Managers and Data Scientists
There are new intermediary roles such as data wranglers, engineers, translators, and data product managers. There was a reference to an insurance company that called them “purple people” – employees with a mix of “red skills” (business analysts) and “blue skills” (leaders who are comfortable with data.)
Some people have argued that tools such as machine learning can close the gap. Both speakers agree that people do a better job than systems, but combining people with systems is the best solution. But as more and more companies become digitized, it’s crucial we solve this problem.
Improving Data Quality – The Embedded Data Manager
Since data is created and used in each department in every company, it makes sense to have an embedded data manager (the tweener) in each department who’s responsible for all data within that team. The speakers suggested this would likely be a part-time role there for performance planning, managing the data supply chain, and integrating data quickly. In essence, it increases the size of the data team by distributing data responsibilities across the business and enhances the outcomes of data initiatives by acting as a ‘translator’ between the business department and data science team.
Data Supply Chain Management
IT departments typically run the early stages of the data supply chain, which involve data gathering, collecting, storing, etc. The latter stages require people who are concerned with usage-related issues. The tweeners work directly with business people, pushing data teams toward thinking about the consumption and use phases of the data supply chain, making sure that everyone understands what the data is supposed to do.
The Data Science Bridge – Driving Collaboration
Tom Redmond shared that when he was at Bell Labs, ATT was fully aware of tensions between line managers and disruptors and, as a result, built a “technology transfer process.” This new notion of the data science bridge is the equivalent of that technology transfer process. With people in the middle of this process, it effectively drives collaboration. People on both sides have to and will learn new skills.
Where Do Tweener Roles Sit in the Organization?
There are many questions about where tweeners sit in an organization: Are they IT people? Are they on the business side? Are they in a Data Science Center of Excellence? Do they report to one part of the organization or a matrix? Do we need a data product manager, a business analyst, and a translator? How do we attract these people? What kinds of backgrounds do they have? How do we know whether they’re doing their jobs well or not? How do we compensate them?
An interesting point of view shared that by sitting in the middle, tweeners won’t (and don’t) get the respect they need in the data science community, but they also don’t get the respect they deserve in the business community.
External Forces Affecting Tweeners
Various external forces are changing the equation. One could argue that citizen development is a way of getting around this “in-between” problem by empowering the business side to do citizen data science. If this notion catches on, it will reduce the need for tweeners. Various low-code, no-code tools make it easier for non-technical people to do sophisticated analytical work. Generative AI will bring this to the next level.
However, Tom Redman pointed out that we haven’t yet solved the data quality problems, increasing the need.
Agile methods dictate interaction between the business and technical sides, requiring tweener facilitation. Poor deployment rate also increases the need, and outsourcing creates even another level of distance between some technical people and some business people.
Intermediaries Can Help in Every Step of the Data Science Process
Tweeners can fit into every step of the data science process, beginning with figuring out the problem to understand the business, followed by preparing data, developing the model, building the model into the process, training, and deployment into the business infrastructure. These are all opportunities to ask: What business connections are required at each step? Who are the intermediaries needed?
Many data science projects (which are primarily intended to solve business problems) fall short because they neglect to put business priorities front and center. Even with a generative AI model, it won’t replace the human need for that first step.
Clarifying Roles for Tweeners
Where do tweeners fall in the organization? One example is to create a data product management function. Vista, a marketing materials company serving small businesses, has a complete product management function, slowly moving the data product managers into product management roles. They’ve already done this for externally-oriented data product managers and eventually expect to do the same for those who work with internal customers. Combining product managers with those who play intermediate data roles may be helpful.
But how do we manage tweeners? They don’t report to lines of business, nor do they report to data science teams. Embedded data managers solve this problem by keeping them within business teams. Dramatic and complex technical evolution with AI and machine learning will only increase the need for alignment.
Another complexity is that there will need to be a career path. Formal education and certification in these roles will eventually follow, creating a pathway for more technically informed and competent business leaders.
Top Takeaways – Organizations Need Tweeners, and It’s Not Easy
The current state in almost every organization today needs tweeners. It’s a challenging role, and making it work will be hard.
There are some clues for what organizations do today, but it requires courage to change with more experimentation. If we continue to “bolt on” data science, democratizing the data will only get more complicated.
It will eventually be everybody’s job to do data-driven, analytics-driven transformation. In the meantime, we need tweeners to make it happen. Some organizations are closer than others. Larger tech companies have people who started in highly data-oriented roles are now progressing through the managerial ranks. But you have yet to find that in many more traditional companies. With the advantages data-driven decision-making brings, they’ll have to adapt soon or get left behind.
The post The Rise of “Purple People:” Tweener Roles in Data Science appeared first on StreamSets.
Read MoreStreamSets
