Help Wanted: Data Analytics Generalist

One of the most persistent debates in the field of Analytics is what a Data Analyst, Engineer, or Scientist should or should not be doing. Accordingly, we’ve seen a shift towards greater specialization of roles within Analytics teams. I believe we have overdone it and it’s hurting Analytics teams.

I’m not surprised by how we got here. Analytics is complex and the topics are deep. The skills required take significant training and there are a variety of functions required. It's also hard to keep up with the rapid pace of innovation and development. We need many (if not all) the specialized roles we have today.

The problem is we have overspecialized and there aren't enough generalist resources on many Analytics teams. This has led to overly long project timelines, reduced context for critical decisions, and suboptimal outcomes.

We need more generalists, not (just) specialists

Adding more generalist team members to Analytics teams will significantly reduce project cost and timelines, the amount of back and forth required, and lead to higher rates of business success and individual satisfaction.

I saw the consequences of overspecialization first hand on a recent project. The goal was to build and deploy a Machine Learning model and involved business, analytics, and technology team members. It played out with:

• 5 months to build the Machine Learning model

• 2 months to build the solution integration

• 3 months of waiting for cross-team prioritization

• 4 months to deploy the new solution with the model

• 2 months to test the results

The project was killed after 18 months because key business requirements were not met. A second iteration of the project was completed largely from scratch in 2 months. Here’s how:

Analytics didn’t just create the ML model. It facilitated the creation of the solution, partnering with Product, Customer Success, and IT.

Analytics owned the project from business concept to deployment, acting as a bridge between Business and Technology stakeholders and requirements. In addition to Analytics' specialized knowledge and skills, the team acted as a "general contractor" for the project as a whole. It helped stitch together the various specialist resources and skills required.

Doing this on a broader scale will significantly reduce project cost and timelines, the amount of back and forth required, and increase the chances of a successful outcome. Not to mention less frustration and greater satisfaction. It will also increase the value you bring to your team and organization.

Some of the hype around Analytics has tempered and there’s more focus on proving clear ROI, especially in today’s business environment.

Don’t get me wrong, some specialization is important and necessary. However, I think we went too far in recent years, leading to a decrease in the impact and value Analytics teams generate. We’re now challenged to change this perception. This is good for organizations, Analytics teams, and Data professionals (including in terms of personal satisfaction and career development).

Expect to see more “Full Stack” Data Scientists in the future.

Also check out this article from Harvard Business Review, which provides a more comprehensive analysis: https://hbr.org/2019/03/why-data-science-teams-need-generalists-not-specialists