When Will AutoML replace Data Scientists? Poll Results and Analysis

In 2015, we asked KDnuggets readers

"When will most expert-level Data Science/Machine Learning tasks - currently done by human Data Scientists - be automated?"

51% of respondents in 2015 poll expected this to happen in 10 years or less.

1. When AutoML will reach expert level?

We asked the same question in the last KDnuggets poll (Feb-Mar 2020), and based on the results from 636 participants, this future is still 5-10 years away.

Fig. 1: When will most expert-level Data Science/Machine Learning tasks - currently done by human Data Scientists - be automated?
58% of readers expect this to happen in 10 years or less, and the median answer is about 8-9 years.

2. AutoML Projects experience

We also asked KDnuggets readers how many projects they have done:

• 0 AutoML projects, 52%
• 1 AutoML project, 17%
• 2-3 projects, 21%
• 4-7 projects, 3.0%
• 8 or more projects, 6.8%

We compared the predictions of "When DS/ML tasks will be automated" from those who had zero AutoML projects with those who have done at least one - see Fig. 2a.

Fig. 2a: "When will most expert-level Data Science/Machine Learning tasks be automated?", comparing No AutoML projects with one or more AutoML projects.

We note that, overall, the curves for no AutoML (black) and one or more AutoML (red) are quite similar. If we look at differences, the people with AutoML experience are more positive about AutoML abilities, and 36% of them predict that AutoML will reach expert level in less than 5 years (vs 30% for people with no AutoML experience). 44% of people with AutoML experience think it will take 5 or more years for AutoML to reach expert level, vs 49% of people with no AutoML experience. Interestingly, a little higher percentage (19.1% vs 17.6%) of people with AutoML experience think that AutoML will NEVER reach human expert level.

These differences are even more pronounced if we compare people with zero AutoML project experience vs those with 2 or more AutoML projects (Fig. 2b).

Fig. 2b: "When will most expert-level Data Science/Machine Learning tasks be automated?", comparing people with No (zero) AutoML projects and those with 2+ AutoML projects

Full data is in Table 1.

Table 1. When will most expert-level Data Science/Machine Learning tasks be automated?" vs "Number of AutoML projects"

We also asked an open-response question: "Which AutoML tools have you used", and there were about 100 different answers given. Of course, there were lots spelling variations, especially for H2O, but the top 10 most frequent answers are below. The field is still very fragmented, as you can see from these results.

1. H2O Automl, 12.0%
2. DataRobot, 6.9%
3. Google Automl (cloud, tables), 6.1%
4. TPOT, 4.6%
5. Azure Automl, 3.5%
6. Rapidminer Auto Model, 2.7%
7. Amazon Sagemaker, 2.4%
8. Automl (Automl 2.9.9), 2.2%
9. SAS Viya autotune, 1.7%

3. Employment type

We also asked respondents their employment type, and the breakdown was

• Industry/Self employed, 68.7%
• Student, 13.7%
• Academia, 5.8%
• Government/non-profit, 4.6%
• Other, 7.2%

Comparing predictions from Industry vs next 2 largest groups Students and Academia/Govt,
we see similar curves. Some notable differences is that larger fraction of students (14%) think that AutoML has already reached expert level, and a lot fewer students think that
AutoML will never reach expert level.

Fig. 3: "When will most expert-level Data Science/Machine Learning tasks be automated?", comparing predictions from Industry, Students, and Academia/Govt.

4. Regions

Finally, the regional breakdown was

• US/Canada, 38.4%
• Europe, 34.0%
• Asia, 16.7%
• Latin America, 3.9%
• Africa/Middle East, 3.8%
• Australia / NZ, 3.3%

We plotted the predictions for the three largest regions - see Fig. 5.

Fig. 4: "When will most expert-level Data Science/Machine Learning tasks be automated?", comparing predictions from US/Canada, Europe, and Asia.

We note that Europe and US/Canada have very similar prediction curves,
while Asian responders are more optimistic about AutoML capabilities.
More of them (14%) think AutoML has already reached expert level, and fewer (9%) think it will never reach them. 73% of Asian responders think AutoML will reach expert level in 10 years, vs 57% of European responders and 53% of US/Canada responders.

Related:

• The Death of Data Scientists – will AutoML replace them?
• AutoML Poll results: if you try it, you’ll like it more
• Automated Machine Learning: How do teams work together on an AutoML project?