Transitioning From AI Teams To AI Squads

Author(s): Dr. Ori Cohen

Originally published on Towards AI.

In the evolving landscape of data-driven organizations, a critical question arises: When should a company transition from a centralized data science team to integrated AI squads?

Organizations embarking on their AI journey would typically begin with a centralized AI team structure. This approach allows companies to “walk before they run” in the complex landscape of AI integration.

Starting with a single data scientist and gradually expanding to a team of experts, this centralized model provides a controlled environment to develop AI capabilities while navigating the disruptive nature of AI adoption.

It addresses the multifaceted challenges AI presents, from knowledge gaps and infrastructure needs to evolving engineering practices and product management paradigms.

A centralized team serves as a focal point for upskilling existing staff, educating business stakeholders, and establishing new workflows and processes. The team will also influence the organization to conduct robust data governance practices and thorough data testing protocols, ensuring a solid foundation for AI development and analytics before considering wider integration.

As the organization learns to seamlessly integrate AI into its business processes, products, and engineering practices, it can incrementally improve its AI operations, reducing time-to-market for models and AI capabilities.

Only when this centralized approach operates flawlessly and AI becomes an integral part of the company’s DNA should the organization consider transitioning to more distributed models like AI squads. This measured progression ensures a solid foundation in AI practices before spreading expertise across the organization, ultimately leading to more successful and sustainable AI adoption.

The transition can begin with a single team featuring one or a few data scientists, or it can evolve into a comprehensive integration where data scientists are embedded within multiple engineering groups.

This article explores the key factors to consider when making this strategic decision. Let’s start by elaborating on AI teams and AI squads.

Centralized Teams vs. Distributed Squads

Centralized AI Teams
AI teams have emerged as a cornerstone of many organizations’ digital transformation efforts. These centralized units bring together specialists in data science, machine learning, and artificial intelligence to tackle complex analytical challenges.

There have been organizational models developed around this concept, such as the “Complicated Subsystem” team from Team Topologies, which describes teams requiring significant mathematical and technical expertise, which is essentially the practical definition of a data science team. Unlike distributed approaches, AI teams concentrate on expertise, fostering deep knowledge sharing and standardization of AI practices across the organization.

The centralized approach offers several advantages, including consistency in methodologies and tools across projects, seamless knowledge sharing among data scientists, and the flexibility to allocate resources to high-priority projects as needed. Data scientists in this model will develop deep expertise in specific techniques, benefiting the entire organization.

However, this centralized model is not without its challenges. As AI teams work on multiple projects simultaneously, they must carefully balance resources and priorities, learn engineering practices, and often collaborate with other departments to fully implement their solutions. This structure, while efficient for developing cutting-edge AI capabilities, can sometimes struggle with seamless integration into broader product development cycles. Nevertheless, for many companies, the AI team remains a crucial hub for driving innovation and leveraging data-driven insights to gain a competitive edge in an increasingly AI-driven business landscape.

Additionally, as the organization grows, the model may become a bottleneck, with increasing project requests leading to longer wait times. Possible communication gaps and missing domain-specific knowledge, which is crucial for certain projects. Implementation delays may occur due to handoffs between the data science team and engineering teams (in the absence of internal engineering practices), and a possible sense of isolation among data scientists, who feel disconnected from the business impact of their work.

Distributed AI Squads
AI squads represent a cross-functional approach to AI and data science integration. Unlike traditional centralized data science teams, these squads typically include a mix of backend developers, frontend developers, DevOps engineers, data scientists, and analysts. The goal is to create self-sufficient units capable of delivering end-to-end AI-driven solutions.

On paper, an AI squad model promises faster delivery through end-to-end ownership, reducing handoffs and accelerating implementation, gaining a deep understanding of specific business areas, and fostering improved synergy between data scientists, engineers, and product managers.

However, this model isn’t without its potential drawbacks, such as business goal misalignment, the development of skill silos with reduced opportunities for data scientists to work on diverse projects, resource duplication across squads, and inconsistent engineering practices. Issues such as efficiency versus agility, i.e., centralized teams could allocate resources more efficiently, but squads could respond more quickly to business needs. The balance between standardization and customization also comes into play, with centralized teams maintaining consistent practices, while squads tailor their approaches to specific business needs.

In the following section, we’ll look at the considerations and the questions you should answer before deciding to transition from a team- model to a squad model.

Key Considerations for Transitioning to AI Squads

The decision to transition to AI squads should be based on a careful evaluation of these factors. Organizations that meet these criteria are likely to benefit from the squad-based approach.

The following factors are listed in order of importance, but your organization’s specific needs may differ, and you may need to prioritize them differently.

Core Business Focus
Q: Is AI central to the company’s core product or business model?

Consider transitioning only if AI is fundamental to the company’s business, and the company is focused on developing AI features end-to-end.

Integration Efficiency
Q: If data scientists are to join engineering teams, will their presence in all daily meetings and engineering discussions be productive?

Transition only after you consider whether full integration adds clear value, adds/reduces efficiency, or if a more flexible collaboration model would be more efficient.

Unified KPIs
Q: Can the organization establish unified Key Performance Indicators (KPIs) for a squad of data scientists and engineers?

Aligned goals and metrics are essential for successful integration and collaboration. Transition only if the organization can establish unified KPIs that align with business objectives, and create a shared goal for a team of data scientists and engineers that provides a clear rationale for merging these functions.

Infrastructure Maturity
Q: Is the organization’s infrastructure sufficiently developed to deploy AI features to production in minimal time?

A mature, robust, and scalable infrastructure is essential for the success of distributed AI teams. I propose to transition only if amature AI infrastructure enables the rapid deployment of AI features, significantly reducing both the time to market and the time to value for AI-driven business solutions.

Organizational Commitment
Q: Is the organization prepared to allocate resources to maintain a strong data science culture even when data scientists are distributed across teams?

A data science team thrives when there’s a solid framework for communication and brainstorming. This structure ensures alignment on objectives, helping all members work toward common business goals while developing a shared understanding of data and challenges. Regular brainstorming sessions encourage creativity and diverse perspectives, leading to innovative solutions that individual members might overlook.

Additionally, such a framework facilitates knowledge transfer, enabling less experienced team members to learn from their peers and fostering skill development. By leveraging shared tools and codebases, teams can avoid duplicated efforts and use resources more efficiently. Furthermore, regular feedback loops promote quality and continuous improvement in work output.

Transitioning to a squad model can lead to a loss of some natural interactions, necessitating the establishment of additional processes to sustain them artificially. To preserve a strong data science culture throughout the organization, it must be committed to fostering ongoing training, knowledge sharing, and community building among our dispersed data science professionals.

Conclusion

The decision to transition from a centralized data science team to integrated AI squads represents a pivotal moment in an organization’s AI journey. While a centralized approach provides a controlled environment for developing AI capabilities, addressing knowledge gaps, and fostering robust governance practices, it can become a bottleneck as project demands grow.

AI squads offer a more agile and cross-functional model, enabling faster delivery and deeper integration with business operations, and can bring significant benefits in terms of agility, innovation, and product development. However, it’s not a one-size-fits-all solution. Organizations must carefully assess their readiness and the potential impact on their data science capabilities. By considering the factors outlined above, companies can make an informed decision about whether and when to make this transition, ensuring that they leverage their data science talent in the most effective way possible.

To make this strategic transition successful, organizations must thoroughly evaluate key factors such as the centrality of AI to their business model, the efficiency of integration within engineering teams, the establishment of unified KPIs, the maturity of their infrastructure, and their commitment to nurturing a strong data science culture. By carefully considering these elements, companies can create a tailored approach that aligns with their specific needs and objectives. Ultimately, the right balance between centralized and distributed models will lead to more successful, sustainable AI adoption, empowering organizations to leverage data-driven insights and innovations in a rapidly evolving landscape.

Finally, A compromise between approaches is to create AI squads only for key product areas. This balanced approach allowed them to leverage the strengths of both models, possibly as an intermediary stage, while mitigating their respective drawbacks. The core team could focus on maintaining standards, driving innovation across the organization, and handling projects that require a broad perspective. Meanwhile, the squads could dive deep into specific business areas, rapidly iterating on product-specific features and maintaining close alignment with business goals.

Dr. Ori Cohen has a Ph.D. in Computer Science with a focus on Artificial Intelligence. He is the author of the ML & DL Compendium and the StateOfMLOps.com.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI