Can India supply her companies with Artificial Intelligence talent?

Originally published in the Economic Times

Quartz recently published a story lamenting the shortage of Indian talent in Artificial Intelligence (AI) and related fields. While the article largely focused on the challenges tech startups face while recruiting AI talent, it’s clear from the conversations we have been having with established enterprises that the supply-demand imbalance for AI talent is as acute across company sizes and industries.

What’s the big problem?

As the Quartz article notes, only 4%...

This is fundamentally a grassroots-level problem. Roles in Data Science and Data Engineering (of which AI is a part) are at the intersection of maths, statistics and programming. This isn’t taught at Indian colleges as part of formal learning. Few academic institutions like the IIT KGP, IIT Kanpur, IIIT Hyderabad have specialised disciplines in Statistics and Information Retrieval (sub-topics of AI). In fact, according to our internal research, less than 2% of professionals who call themselves data scientists or data engineers have a PhD in AI-related technologies.

Serious Business

This shortage of talent is further exacerbated by the global blurring of lines between tech and non-tech industries. In addition to the technology companies, Advanced Data Science and Data Engineering skills today are sought after by traditionally non-tech industries like Retail, Financial Services, Healthcare, Logistics and Pharmaceutical. As a result, competition for talent has dramatically shot up, even as supply has barely budged.

Is it any wonder then that AI-driven companies are struggling to hire? In fact, the Quartz article quotes extensively from company founders on how they spot talent. A common thread running through the narrative is that most companies are flipping their hiring from inbound to outbound: taking a proactive, data-driven approach to finding and engaging talent instead of trying to get candidates to apply. This is also consistent with what we are seeing in talent acquisition teams at larger enterprises hiring for AI capabilities and skills.

What can companies do to overcome this crunch?

First thing companies can do is to develop a data-backed understanding of the larger ecosystem. Exactly what companies and industries can they tap into?

A cursory look at the industry-wide spread of Data Scientists shows that IT Services, Computer Software and Financial Services companies are the largest employers today and are mostly likely hiring within these segments. But a deeper look into “Others” reveals that Pharmaceuticals, Healthcare and Retail companies have been scaling their Data Science and Engineering teams rapidly over the last year.

Second, when you consider talent supply by location, Bangalore dramatically outranks its peers in the country. For companies not located in Bangalore or NCR, it’s wise to put some thought into a talent strategy that involves attracting talent from one of these cities. For those in Bangalore, however, expect stiffer competition and higher costs as demand for AI-related skills is significantly higher than what we are seeing in other cities.

Finally, how can companies know in advance what their hiring efforts, costs and timelines for a given role are going to be? Is there a single metric employers can track?

To answer precisely this question, we recently launched the Talent Supply Index at Belong. The TSI divides the total number of relevant people for a role in a given market by the total number of active opportunities for that role. For example, if there are 1000 Data Scientist opportunities against only 800 relevant Data Scientists, the role scores 0.8 on the TSI. The closer the role tends to zero, the more supply-constrained is the market. In fact, a score of less than 1 indicates it’s a supply-negative market.

The extreme shortage in quality talent for Data Science and Data Engineering role is illustrated in the chart above. A commonly hired for role like a Java Engineer is compared to five popular Data Science/Engineering roles to highlight the point. Core AI roles related to NLP, Deep Learning, Machine Learning today are clearly in a supply-negative market.

For talent acquisition leaders, implications in such a case range across hiring timelines, costs, need for a stronger employer brand, more effective candidate engagement. But there are three things we’d like to call out.

Align expectations across business

Companies need to set their talent expectations based on a clear understanding of the ecosystem. Many a time, we see businesses start their search looking for the “purple squirrels,” but eventually go back to redefining their search criteria after much time and effort has gone in. Not only does this unnecessarily escalate hiring cycles, but also causes bad candidate experiences.

A proactive, data-backed approach to hiring

For the first time, companies today have access to data and technology to make more informed decisions around their talent strategy. For Data scientists for example, one should consider looking at mapping the entire ecosystem to first understand how many people exist in the universe for each of the different skills. Also, since a lot of these people might not be on traditional job portals, companies should be proactive and look for these folks in places where they’re likely to be present. One can look at authors of published works (via Google Scholar/JStor/PubNub) or patents filed (via Google Patents) in Data Science related topics. One can also look at Data Science or Data Engineering focused public mailing lists such as the Apache Hadoop/Python Mailing lists (there are more than 10000+ such lists).

Build your network of mentors for your team

One of the most effective strategies that organizations can employ is to develop a strong pool of Data Science/Engineering mentors. Mentors can help in various ways, from spreading the good word about your company’s work to their network to helping in evaluating and mentoring your teams.

The rapidly changing business and talent landscape requires companies to start investing in talent strategy and work closely with business to identify and target the right talent for their organizations. We believe that companies embracing this approach are more likely to accelerate further towards their mission versus those shooting in the dark by not leveraging the power of data.

If you're looking to build a great data science team (or any other team for that matter), join enterprises like Cisco, Thoughtworks, UnitedHealthGroup, Fractal Analytics or fast growing startups like Postman, Xola, Lendinkart and many many others and sign up for Belong Hire or Belong Experts.