This week Workday announced intent to acquire HiredScore, a leading provider of AI-based matching tools for recruiting (called “talent orchestration”). While it wasn’t discussed much in the earnings call, this deal is a big positive for Workday and could have many implications for the HR Tech market.
Let me explain. (I have not been briefed by Workday yet, so more information will come as I learn more.)
Right now there is a massive marketplace war for high-powered AI-based recruiting tools (estimated at $30.1 billion). Historically dominated by applicant tracking systems (ATS), this market provides essential technology to help every company grow.
The ATS market, which is more than 25 years old, has been rapidly transformed with high-powered AI tools that help with candidate matching, search, skills inference, and sourcing. And now that AI tools are readily available, these systems are becoming big data platforms loaded with billions of employee profiles, running complex AI models to help match people to jobs, projects, and gigs.
Most ATS vendors (including Workday) have slowly extended into this space through matching. The original idea of a resume parser (software that reads a resume and scores it against a job description) has evolved into complex text analysis and AI-powered inference technology, forcing ATS vendors to invest.
As the ATS vendors enhance their AI capabilities, a parallel universe of AI-first Talent Intelligence vendors emerged. These vendors, like Eightfold, Gloat, Beamery, Phenom, Seekout, Skyhive, Retrain, and Techwolf are building skills-centric big data platforms to match people to jobs, gigs, and mentors. These systems do much more than rate matches: they identify skills, find adjacent skills, match people to careers, find mentors, and more. They are essentially open big-data AI platforms built on vector databases that can be used for many enterprise apps (job architecture design, skills planning, internal mobility, pay equity analysis, etc.). In many ways they represent the future of HR Tech.
(Read our Talent Intelligence Primer for more.)
As the Talent Intelligence vendors grow, they start to deliver “HCM-threatening” platforms that impinge on the HCM “System of Record” idea. If you have all your employees, candidates, alumni, and prospects in Eightfold, Phenom, Seekout, or Gloat, for example, Workday or SAP look like a tactical payroll and workflow management system. (ServiceNow also understands this, and is building talent intelligence into its workflow platform.)
Up until now the big HCM vendors like Workday, Oracle, and SAP have struggled to build these new systems, largely because their original architectures were not AI-based. So they’ve attracted customers with offerings like the Workday Skills Cloud or SAP Opportunity Marketplace that aren’t fully completed yet. We have talked with dozens of Workday Skills Cloud customers, for example, and they see it as an important “skills system of record,” but its real AI matching and inference capabilities have been limited.
Along comes HiredScore, a well respected AI-based matching system with 150 employees and 40+ seasoned AI engineers in Israel. These folks are experts at candidate matching (quite a complex problem), and they’ve built a very innovative “orchestration” system to help line managers coordinate activities with HR business partners and recruiters (more on this later). While I’m sure they’ll continue to build out HiredScore, they can also contribute to Workday’s overall talent intelligence offering, improving the entire system – including the Skills Cloud, Workday Learning, Workday’s Talent Marketplace.
As large as the recruiting software market is, the market for internal career tools, talent mobility, skills inference, and corporate learning is five times bigger. This acquisition gives Workday a shot in the arm to accelerate its entire AI platform strategy. (As the Identified acquisition did back in 2014. Identified was the roots of the Workday Skills Cloud.)
Market Implications Of This Move
This move could change the market for HR software in a few significant ways.
First, Workday Recruiting customers will be thrilled. Workday’s ATS now benefits from a first class matching and candidate scoring solution. This helps Workday compete with the bigger ATS players and gives Workday a new revenue source as they sell HiredScore to the existing 4,000+ Workday ATS customers. (Similar to the Peakon acquisition in Employee Experience.) And the talent orchestration features (kind of like a “staffing copilot”) gives Workday a very unique feature set.
Second, this forces Workday’s talent intelligence partners to step up their game. Remember when Apple acquired Dark Sky, the most compelling micro-weather app on the market? Once they integrated it into Apple’s other apps, the market for third party weather apps went away. Workday could limit its partner network to avoid letting HiredScore competitors into the ecosystem.
Third, this forces HCM vendors to accelerate their AI. Since HiredScore is such a well-respected product (every client we talk with adores it), it will become part of Workday demos and sales proposals quickly. Workday’s HCM competitors will start scratching around to find a similarly mature AI vendor to acquire. And that could kick off another round of acquisitions, similar to the frenzy that took place in the mid 2010s.
Finally, there’s one more scenario, and I give this good odds. Not to be outdone by Workday, the Talent Intelligence vendors may just expand their ATS capability and decide to go “full stack.” I wouldn’t be surprised to see this happen.
Why Is AI-Based Candidate Matching So Important
Why is this technology so important? Well if you’ve ever tried to recruit on Indeed or LinkedIn, you know why. The quality and reliability of “candidate matching technology” is a lynchpin of a talent platform. Just as Google Search crushed Yahoo, Excite, and Inktomi, a powerful next-gen matching tool adds an enormous amount of value. Not only does it speed talent acquisition, it fuels all the internal mobility, career portals, skills, and eventually learning and pay systems.
Why do I say this? A “match” is a sophisticated problem. Unlike a Google search which looks at text and traffic, when you search for a person to fill a role you have to think about dozens of complex relationships. What are this person’s skills and capabilities? What are their credentials or certifications? Who else are they connected with? How likely will they fit into the job, role, and company? What is the impact of their industry experience? What tools and technologies do they understand?
And it gets much more complex. The Heidrick Navigator platform (built on Eightfold), uses AI to assess functional skills for management and leadership, identifies a person’s “ability to drive results,” and more. This important application of AI powers many of the most important decisions we make in business.
That’s why the Talent Intelligence space is growing so fast. As of this week there are more than 1,800 Director or VPs of “Talent Intelligence” in LinkedIn, and that number is up almost six-fold from one year ago.
Can Workday take the lead in this emerging space? It’s impossible to tell at this point, but the horses have left the gate and the race is on. This deal sets the players in the right lanes and feels like the earthquake to shake things up.
滴滴出行选用NICE，以提供基于实时 AI 的个性化服务NICE has partnered with DiDi Global to enhance customer and employee experiences through its cloud-based Workforce Management (WFM) and Employee Engagement Manager (EEM) solutions. This collaboration aims to streamline DiDi's global contact center operations, improving operational efficiency and customer satisfaction with AI-driven forecasting and scheduling. The implementation of NICE's solutions facilitates real-time management and self-scheduling for agents, boosting employee engagement and operational efficiency. DiDi's choice of NICE highlights the importance of advanced, flexible technology in supporting the dynamic needs of modern, app-based transportation services.
领先的移动出行平台通过利用 NICE 的客户体验 AI 技术，使其员工能够提供轻松且高效的客户服务体验
新泽西州霍博肯-NICE (纳斯达克: NICE) 今日宣布，滴滴出行已经选用了 NICE 劳动力管理 (WFM) 和员工参与管理 (EEM) 作为其云端创新技术的一部分。滴滴现在可以全面预测、规划和管理其全球客户联系中心的运作；同时提升运营效率和员工的参与度，并确保客服代表能够在首次通话中解决问题。Betta作为全球最大的 WFM 客户群之一的支持者，在实施过程中与 NICE 价值实现服务携手合作，负责执行集成，并在多国提供咨询、培训和支持服务。
滴滴出行寻求一种能够满足其核心业务、功能及技术需求，并能够随公司成长而扩展的劳动力管理解决方案。NICE WFM 结合了 AI 技术与灵活性，能够满足跨多个大洲、具有特定区域特色的运营需求，这不仅成本效益高，而且精确度高，确保维持最佳的服务水平。通过精准预测，确保在合适的时间有合适技能的代理人，从而大幅提升客户满意度。
通过引入 NICE EEM，可以实时解决人员配置需求，使得客服代理能够自我调节工作时间表，从而增强员工参与度和工作满意度。此外，利用智能日内自动调整功能，能够主动地进行调整，预防问题的发生。
滴滴出行国际客户体验执行总监 Caio Poli 表示：“基于多个考量因素，NICE 显然是我们的首选。我们寻找的是一个顶尖的云端劳动力管理解决方案，能够使我们的全球运营在保证运营效率和员工参与度的同时，提供卓越的客户体验。NICE 的智能日内自动化功能给我们留下了深刻印象，我们的选择是基于 AI 驱动的策略以及云技术的速度和灵活性。”
NICE 美洲总裁 Yaron Hertz 表示：“随着滴滴持续全球扩张，NICE 很高兴有机会为这家数字时代最具创新和活力的应用型运输公司之一提供服务。我们相信，通过采用 NICE 的 AI 驱动预测和机器学习来进行最适合的调度安排，对于联系中心和员工而言，这将有助于推动滴滴的未来发展。”
滴滴出行公司是一个领先的移动技术平台，它在亚太地区、拉丁美洲及其他全球市场提供一系列基于应用的服务，包括网约车、叫车服务、代驾以及其他共享出行方式，还涵盖某些能源和车辆服务、食品配送和城市内部货运服务。滴滴为车主、司机和配送伙伴提供灵活的工作和收入机会，致力于与政策制定者、出租车行业、汽车行业及社区合作，利用 AI 技术和本地化智能交通创新解决全球的交通、环境和就业挑战。滴滴力图为未来城市构建一个安全、包容和可持续的交通与本地服务生态系统，以创造更好的生活体验和更大的社会价值。更多信息，请访问：www.didiglobal.com
借助 NICE (纳斯达克: NICE)，全球各地不同规模的组织现在可以更容易地创造卓越的客户体验，同时满足关键的业务指标。作为世界领先的云原生客户体验平台 CXone 的提供者，NICE 是 AI 驱动自助服务和代理辅助客户体验软件领域的全球领导者，服务范围超出了传统的联系中心。超过 25,000 个组织在超过 150 个国家，包括 85 家以上的财富 100 强公司，都选择与 NICE 合作，以改造并提升每一次客户互动。www.nice.com
商标说明：NICE 和 NICE 标志是 NICE Ltd. 的商标或注册商标。所有其他标志属于它们各自的所有者。NICE 商标的完整列表，请访问：www.nice.com/nice-trademarks。
Is DEI Going to Die in 2024?Josh Bersin 的文章讨论了 2024 年多元化、公平与包容（DEI）项目所面临的重大挑战和批评，特别强调了 "反觉醒 "评论家的攻击和克劳迪娜-盖伊（Claudine Gay）从哈佛辞职的事件。报告探讨了多元包容计划在当前的文化战争中扮演的角色、人们对它的看法以及法律挑战对多元包容计划招聘和投资的影响。尽管存在这些挑战，贝尔辛还是强调了发展型企业的实际商业利益，展示了成功的战略以及将发展型企业融入业务而不仅仅是人力资源的重要性。他认为，应将重点转向在所有业务部门嵌入包容、公平薪酬和开放讨论的原则，并指出，未来的企业发展指数至关重要，但需要适应和领导层的承诺才能茁壮成长。
Is DEI Going to Die in 2024? By Josh Bersin
For anyone working in Diversity, Equity, and Inclusion (DEI), it is safe to say that it has been a tough start to 2024. For a while now, there has been a concerted attack on DEI programs, with ‘anti-woke’ commentators and public figures querying their value, worth, and even existence.
Those attacks increased enormously in 2024 with the resignation of Claudine Gay from Harvard. While the call to resign was supposedly related to plagiarism, one can’t help but feel that her position as a leading DEI advocate also fuelled the demand.
It means that DEI has come under increased and sustained fire, and despite the many benefits provided by a good DEO program – to both employer and employee – there is a feeling that 2024 could be the year that DEI fades away. How likely is this to happen, and what would the impact be if it did?
DEI and the culture wars
Anyone living and working in the US (or most other countries worldwide) over the past few years will have likely heard of the culture wars. Brought on by declining trust in institutions, growing inequalities, and the proliferation of technology, the culture wars involve opposing social groups seeking to impose their ideologies.
All manner of things has been caught up in this, from what’s on the curriculum at schools to taking a knee at sporting events and from definitions of what constitutes a woman to allegations of tokenism in the workplace. DEI has played an unwitting but central part in the culture wars.
There’s a perception that DEI programs are ‘woke’ and prioritize ethnicity and gender over achievement and ability. In August of 2023, an attorney filed (and won) a lawsuit against a VC firm that gives grants to black entrepreneurs. Similar suits have been filed against firms with diversity hiring programs, scholarships, and internships.
The resignation of Claudine Gay has reinvigorated the commentary around DEI programs. Josh Hammer, a conservative talk show host and writer, wrote on the social media platform X that taking down Dr. Gay was a “huge scalp” in the “fight for civilizational sanity. ” It was described as “a crushing loss to DEI, wokeism, antisemitism & university elitism,” by conservative commentator Liz Wheeler, and the “beginning of the end for DEI in America’s institutions,” by the conservative activist Christopher Rufo, who had helped publicize the plagiarism allegations against Claudine Gay.
When something is as consistently criticized and devalued as DEI programs have been, a toll is inevitably taken. That is certainly indicated by the latest hiring data for DEI professionals. According to data from labor market analytics company Lightcast, hiring for DEI positions in the US is down by 48% year over year, in the middle of an economic boom. Clearly, DEI investments are under attack.
And when you look at companies doing layoffs, DEI jobs are frequently high on the list of jobs to cut. I even heard a recent podcast with four well-known venture capitalists – three agreed that “doing away with DEI programs” was top on their list.
The value of DEI
Given this criticism of DEI programs, one could be forgiven for thinking such programs carry no value to HR and the wider business. Yet many companies invest in DEI programs, and the value is high in almost every case I come across.
Our Elevating Equity research in 2022 and 2023 found companies focus on diversity and inclusion for very pragmatic reasons, including:
An inclusive hiring strategy broadens and deepens the recruiting pool.
An inclusive leadership strategy drives a deeper leadership pipeline.
An inclusive management approach helps attract diverse customers and markets.
An inclusive board drives growth and market leadership. (proven statistically)
An inclusive supply chain program improves sustainability of the supply chain.
An inclusive culture creates growth, retention, and engagement in the employee base.
Organizations are not prioritizing DEI programs because they are woken or as a box-ticking exercise. They do so because DEI provides real and tangible business benefits. Workday, one of the most admired HR technology companies in the market, has pioneered DEI internally and through its products, and the company has outgrown and outperformed its competitors for years. Their product VIBE, an analytics system designed for this purpose, shows intersectionality, and helps companies set targets and find inequities in leadership, hiring, pay, and career development.
But some law firms have posited that these types of programs are illegal – is there a case to answer?
In response, it’s important to consider the massive and complex pay equity problem. Until the last few years, most companies had no problem paying people in very idiosyncratic ways. The Josh Bersin Company looked at leadership, succession, and pay programs worldwide last year and found that there are massive variations in pay with no clear statistical correlation in most larger companies.
This problem is called “pay equity,” and when you look at pay vs. gender, age, race, nationality, and other non-performance factors, most companies find problems. Is this a “DEI” program?
When we looked at pay equity in detail last year, we found that only 5% of companies have embarked on a strategic equity analysis. While most companies do their best to keep pay consistent with performance, these studies always find problems. Would it be considered illegal to analyze pay by race or nationality and then fix the disparities?
The future of DEI
DEI is undoubtedly a complex issue, and many organizations will be uncertain about the best course of action. Despite the current wave of criticism, there has been vast investment in DEI strategy over recent years, and business leaders are highly unlikely to let that fade away.
Despite the anti-woke movement, political debates, and the inability of Harvard, Penn, and other universities to speak clearly on these topics, businesses will not stop. Affirmative Action was not created to discriminate; it was designed to reduce discrimination. At the University of California, where Affirmative Action was halted in 1995, studies found that earnings among African American STEM graduates decreased significantly. So, one could argue that they were making a real difference.
DEI will not die – it is far too important for that to happen. However, it’s time to do away with the “DEI police” in HR and focus on embedding the principles of inclusion, fair pay, and open-minded discussions across all business units. Senior leaders must take ownership of this issue.
In the early 2000s, companies hired Chief Digital Officers to drive digital technology implementation, ideas, and strategies. As digital tools became commonplace, the role went away. We may be entering a period where the Chief Diversity Officer has a new role: putting the company on a track to embrace inclusion and diversity in every business area and spending less time pushing the agenda from a central group.
In every interview we conduct on this topic, we see overwhelming positive stories from various DEI strategies. Each successful company frames DEI as a business rather than an HR strategy. While HR-centric DEI investments are shrinking, it’s more like them migrating into the business where they belong.
Claudine Gay的辞职再次引发了对DEI项目的广泛讨论。保守派脱口秀主持人和作家Josh Hammer在社交媒体平台X上表示，击败Gay博士是“为文明理智而战的一大胜利”。保守派评论员Liz Wheeler称之为“对DEI、觉醒主义、反犹太主义及大学精英主义的沉重打击”，而保守派活动家Christopher Rufo则称这是“DEI在美国机构中走向终结的开始”。
面对这一问题，我们不得不考虑到复杂且广泛的薪酬公平问题。直到最近几年，大多数公司在个性化支付薪酬方面并未遇到太大问题。Josh Bersin Company去年对全球的领导力、继承计划和薪酬计划进行了研究，发现在许多大公司中，薪酬存在巨大差异，且大多没有明显的统计相关性。
How Generative AI Adds Value to the Future of Work
Authors: Dr. Ted Liu, Carina Deng, Dr. Kelly Monahan
Generative AI’s impact on work: lessons from previous technology advancements
In this study, we provide a comprehensive analysis of the initial impact of generative AI (artificial intelligence) on the Upwork marketplace for independent talent. Evidence from previous technological innovations suggests that AI will have a dual impact: (1) the displacement effect, where job or task loss is initially more noticeable as technologies automate tasks, and (2) the reinstatement effect, where new jobs and tasks increase earnings over time as a result of the new technology. Take for example the entry of robotics within the manufacturing industry. When robotic arms were installed along assembly lines, they displaced some of the tasks that humans used to do. This was pronounced in tasks that were routine and easy to automate. However, new tasks were then needed with the introduction of robotics, such as programming the robots, analyzing data, building predictive models, and maintaining the physical robots. The effects of new technologies often counterbalance each other over time, giving way to many new jobs and tasks that weren’t possible or needed before. The manufacturing industry is now projected to have more jobs available as technologies continue to advance, including Internet of Things (IoT), augmented reality, and AI, which transform the way work is completed. The issue now at hand is ensuring enough skilled workers are able to work alongside these new technologies.
While this dynamic of displacement and reinstatement generally takes years to materialize, as noted above in the manufacturing example, the effects of generative AI may be taking place already on Upwork. For the platform as a whole, we observe that generative AI has increased the total number of job posts and the average spend per new contract created. In terms of work categories, generative AI has reduced demand in writing and translation, particularly in low-value work, while enhancing earnings in high-value work across all groups. In particular, work that relies on this new technology like Data Science and Analytics are reaping the benefits. The report highlights the importance of task complexity and the skill-biased nature of AI's impact. Skills-biased technology change is to be expected as the introduction of new technologies generally favors highly skilled workers. We observe this on our platform as high-skill freelancers in high-value work are benefiting more, while those in low-value work face challenges, underscoring the need for skilling and educational programs to empower freelancers to adapt and transition in this evolving work landscape.
Understanding the lifecycle of work on Upwork and the impact of gen AI
Generative AI has a growing presence in how people do their work, especially since the public release of ChatGPT in 2022. While there’s been extensive discussion about the challenges and opportunities of generative AI, there is limited evidence of such impact based on transaction data in the broader labor market. In this study, we use Upwork’s platform data to estimate the short-term effects of generative AI on freelance outcomes specifically. The advantage of the Upwork platform is that it is in itself a complete marketplace for independent talent, as we observe the full life cycle of work: job posts, matching, work execution, performance reviews, and payment. Few other instances exist where a closed-system work market can be studied and observed. Thus, the results of this study offer insights into not only the online freelance market, but also the broader labor market.
How technological progress disrupts the labor market is not a new topic. Acemoglu and Restrepo (2019) argue that earning gain arises from new tasks created by technological progress, which they term the “reinstatement effect,” even if the automation of certain tasks may have a displacement effect in the labor market initially. What this means is that there may be a dynamic effect going on: the displacement effect (e.g., work loss) may be more noticeable in the beginning of a new technology entry, but as new jobs and tasks are being created, the reinstatement effect (e.g., rates increase, new work) will begin to prevail. In the broader labor market, such dynamics will likely take years to materialize. But in a liquid and active independent work marketplace like Upwork, it’s possible that we’re already observing this transition happening.
Existing studies such as this provides a useful conceptual framework to think about the potential impact of generative AI. It’s likely that in the short term, the replacement of generative AI will continue to be more visible, not just at Upwork, but also in the broader labor market. Over time and across work categories, however, generative AI will likely spur new tasks and jobs, leading to the reinstatement effect becoming stronger and increasing rates for those occupations with new tasks and a higher degree of task complexity. We’ve already seen evidence of new demand as a result of gen AI on our Upwork platform, with brand new skill categories like AI content creator and prompt engineer emerging in late 2022 and early 2023. We test this hypothesis of both work displacement and reinstatement, and provide insights into how generative AI affects work outcomes.
Impact of generative AI on work
To understand the short-term impact of generative AI on the Upwork freelance market, we capitalize on a natural experiment arising from the public release of ChatGPT in November 2022. Because this release was largely an unanticipated event to the general public, we’re able to estimate the causal impact of generative AI. The essential idea behind this natural experiment is that we want to compare the work groups affected by AI with the counterfactual in which they are not. To implement this, we use a statistical and machine-learning method called synthetic control. Synthetic control allows us to see the impact that an intervention, in this case, the introduction of gen AI, has on a group over time by comparing it to a group with similar characteristics not exposed to the intervention. The advantage of this approach is that it allows us to construct reasonably credible comparison groups and observe the effect over time.
The units of analysis we use are work groups on the Upwork platform; we analyze variables such as contract number and freelancer earnings. Instead of narrowly focusing on a single category like writing, we extend the analysis to all the major work groups on Upwork. Moreover, we conduct additional analysis of the more granular clusters within each major group. The synthetic control method allows for flexibility in constructing counterfactuals at different levels of granularity. The advantage of our comprehensive approach is that we offer a balanced view of the impact of generative AI across the freelance market.
Generative AI’s short-term impact on job posts and freelancer earnings
Looking at the platform as a whole, we observe that generative AI has increased the total number of job posts by 2.4%, indicating the overall increased demand from clients. Moreover, as shown in Figure 1, for every new job contract, there is an increase of 1.3% in terms of freelancer earnings per contract, suggesting a higher value of contracts.
Figure 1 Effect of Generative AI on Freelancer Earning per Contract
The Upwork platform has three broad sectors: 1. Technological and digital solutions (tech solutions); 2. Creative & outreach; 3. Business operations and consulting. We have observed both positive and negative effects within each of the sectors, but two patterns are worth noting:
The reinstatement effect of generative AI seems to be driving growth in freelance earnings in sectors related to tech solutions and business operations. In contrast, within the creative sector, while sales and marketing earnings have grown because of AI, categories such as writing and translation seem disproportionately affected more by the replacement effect. This is to be expected due to the nature of tasks within these categories of work, where large language models are now able to efficiently process and generate text at scale.
Generative AI has propelled growth in high-value work across the sectors and may have depressed growth in low-value work. This supports a skills-biased technology change argument, which we’ve observed throughout modern work history.
More specifically and within tech solutions, data science & analytics is a clear winner, with over 8% of growth in freelance earnings attributed to generative AI. This makes sense as the reinstatement effect is at work; new work and tasks such as prompt engineering have been created and popularized because of generative AI. Simultaneously, while tools such as ChatGPT automate certain scripting tasks (therefore leading to a replacement effect), it mainly results in productivity enhancements for freelancers and potentially leads to them charging higher rates and enjoying higher overall earnings per task.
In terms of contracts related to business operations, we observe that accounting, administrative support, and legal services all experience gains in freelance earnings due to generative AI, ranging from 6% to 7%. In this sector, customer service is the only group that has experienced reduced earnings (-4%). The reduced earnings result for customer service contracts is an example of the aggregate earnings outcomes of AI, related to the study by Brynjolfsson et al (2023), who find that generative AI helps reduce case resolution time at service centers.
A potential outcome of this cut in resolution time is that service centers will need fewer workers, as more tasks can be completed by a person working alongside AI. At the same time, the reinstatement effect has not materialized yet because there are no new tasks being demanded in such settings. This may be an instance where work transformation has not yet been fully realized, with AI enabling faster work rather than reinventing a way of working that leads to new types of tasks. A contrasting case is the transformation that happened with bank tellers when ATMs were introduced. While the introduction of these new technologies resulted in predictions of obsolete roles in banks, something different happened over time. Banks were able to increase efficiency as a result of ATMs and were able to scale and open more branches than before, thereby creating more jobs. In addition, the transactional role of a bank teller became focused on greater interpersonal skills and customer relationship tasks.
When taken together, the overall gains in such business operations work on Upwork are an encouraging sign. These positions tend to require relatively intensive interpersonal communication, and it seems the short-term effects of generative AI have helped increase the value of these contracts, similar to what we saw in the banking industry when ATMs were introduced.
As of now, the replacement effect of AI seems more noticeable in creative and outreach work. The exception is sales and marketing contracts, which have experienced a 6.5% increase in freelance earnings. There is no significant impact yet observed on design. For writing and translation, however, generative AI seems to have reduced earnings by 8% and 10% respectively. However, as we will discover, task complexity has a moderating effect on this.
High-value work benefit from generative AI, upskilling needed for low-value work
Having discussed the overall impact of generative AI across categories, we now decompose the impact by values. The reason we’re looking at the dimension of work value is that there may be a positive correlation between contract value and skill complexity. Moreover, skill complexity may also be positively correlated with skill levels. Essentially, by evaluating the impact of AI by different contract values, we can get at the question of AI's impact by skill levels. This objective is further underscored by a discrepancy that sometimes exists in the broader labor markets – a skills gap between demand and supply. It simply takes time for upskilling to take place, so it’s typical for demand to exceed supply until a more balanced skilled labor market takes place. It is worth noting, however, freelancers on the Upwork platform seem more likely than non-freelancers to acquire new skills such as generative AI.
For simplicity, let’s assume that the value of contracts is a good proxy for the level of skill required to complete them. We’d then assume that high-skill freelancers typically do high-value work, and low-skill freelancers do low-value work. In other words, our goal is also to understand whether the impact of generative AI is skills-biased and follows a similar pattern from what we’ve seen in the past with new technology disruptions. Note that we’re focusing on the top and bottom tails of the distribution of contract values, because such groups (rather than median or mean) might be most susceptible to displacement and/or reinstatement effects, therefore of primary concern. We define high-value (HV) work as those with $1,000 or more earnings per contract. For the remaining contracts, we focus on a subset of work as low-value (LV) work ($251-500 earnings).
Figure 2 shows the impact of AI by work value, across groups on Upwork. As we discussed before, writing and translation work has experienced some reduction in earnings overall. However, if we look further into the effect of contract value, we see that the reduction is largely coming from the reduced earnings from low-value work. At the same time, for these two types, generative AI has induced substantial growth in high-value earnings – the effect for translation is as high as 7%. We believe the positive effect on translation high-value earning is driven by more posts and contracts created. In the tech solutions sector, the growth in HV earnings in data science and web development is also particularly noticeable, ranging from 6% to 9%. Within the business solutions sector, administrative support is the clear winner.
There are two takeaways from this analysis by work value. First, while we’re looking at a sample of all the contracts on the platform, it’s possible that the decline of LV work is more than made up for by the growth of HV work in the majority of the groups. In other words, except for select work groups, the equilibrium results for the Upwork freelance market overall seem to be net positive gains from generative AI. Second, if we assume that freelancers with high skills (or a high degree of skill complexity) tend to complete such HV work (and low-skill freelancers do LV work), we observe that the impact of generative AI may be biased against low-skill freelancers. This is an important result: In the current discussion of whether generative AI is skill-based, there exists limited evidence based on realized gains and actual work market transactions. We are one of the first to provide market-transaction-based evidence to illustrate this potentially skill-biased impact. Finally, additional internal Upwork analysis finds that independent talent engaged in AI-related work earn 40% more on the Upwork marketplace than their counterparts engaged in non-AI-related work. This suggests there may be additional overlap between high-skill work and AI-related work, which can further reinforce the earning potential of freelancers in this group.
Case study: 3D content work
To illustrate the impact of generative AI in more depth, we have conducted a case study of Engineering & Architecture work within the tech solutions sector. The reason is that we want to illustrate the potentially overlooked aspects of AI impact, compared with the examples of data science and writing contracts. This progress in generative AI has the potential to reshape work in traditional areas like design in manufacturing and architecture, which rely heavily on computer-aided design (CAD) objects, and newer sectors such as gaming and virtual reality, exemplified by NVIDIA's Omniverse.
Based on activities on the Upwork platform, we see that there is consistent growth of job posts and client spending in this category, with up to 12% of gross service value growth year over year in 2023 Q3, and over 11% in job posts during the same period. Moreover, applying the synthetic control method, we show a causal relationship between gen AI advancements and the growth in job posts and earnings per contract. More specifically, there is a significant increase in overall earnings because of AI, an average 11.5% increase. Additionally, as shown by Figure 3, the positive effect also applies to earning per contract. This indicates a positive impact on freelancer productivity and quality of work, due to the fact that we’re measuring the income for every unit of work produced. This suggests that gen AI is not just a facilitator of efficiency but also enhances the quality of output.
Figure 3 Effect of Generative AI on Freelancer Earning per Contract in EngineeringIn a traditional workflow to create 3D objects without generative AI, freelancers would spend extensive time and effort to design the topology, geometry, and textures of the objects. But with generative AI, they can do so through text prompts to train models and generate 3D content. For example, this blog by NVIDIA’s Omniverse team showcases how ChatGPT can interface with traditional 3D creation tools.
Thus, the positive trajectory of generative AI in 3D content generation we see is driven by several factors. AI significantly reduces job execution time, allowing for higher productivity. It facilitates the replication and scaling of 3D objects, leading to economies of scale. Moreover, freelancers can now concentrate more on the creative aspects of 3D content, as AI automates time-consuming and tedious tasks.
This shift has not led to a decrease in rates due to the replacement effect. In fact, this shift of workflow may create new tasks and work. We will likely see a new type of occupation in which technology and humanities disciplines converge. For instance, a freelancer trained in art history now has the tools to recreate a 3D rendering of Japan in the Edo period, without the need to conduct heavy coding. In other words, the reinstatement effect of AI will elevate the overall quality and value proposition of the work, and ultimately enable higher earning gains. This paradigm shift underscores generative AI's role in not just transforming work processes but also in creating new economic dynamics within the 3D content market. Fortunately, it seems many freelancers on Upwork are ready to reap the benefits: 3D-related skills, such as 3D modeling, rendering, and design, are listed among the top five skills of freelancer profiles as well as in job posts.
A dynamic interplay: task complexity, skills, and gen AI
Focusing on the Upwork marketplace for independent talent, we study the impact of generative AI by using the public release of ChatGPT as a natural experiment. The results suggest a dynamic interplay of replacement and reinstatement effects; we argue that this dynamic is influenced by task complexity, suggesting a skill-biased impact of gen AI. Analysis across Upwork's work sectors shows varied effects: growth in freelance earnings in tech solutions and business operations, but a mixed impact in the creative sector. Specifically, high-value work in data science and business operations see significant earnings growth, while creative contracts like writing and translation experience a decrease in earnings, particularly in lower-value tasks. Using the case study of 3D content creation, we show that generative AI can significantly enhance productivity and quality of work, leading to economic gains and a shift toward higher-value tasks, despite initial concerns of displacement.
Acemoglu and Restrepo (2019) argue that the slowdown of earning growth in the United States the past three decades can partly be explained by new technologies’ replacement effect overpowering the reinstatement effect. But with generative AI, we’re at a point of completely redefining what human tasks mean, and there may be ample opportunities to create new tasks and work. It's evident that while high-value types of work are being created, freelancers engaged in low-value tasks may face negative impact, possibly due to a lack of skills needed to capitalize on AI benefits. This situation underscores the necessity of supporting freelancers not only in elevating their marketability within their current domains but also in transitioning to other work categories.
To ensure as many people as possible benefit, there’s an imperative need to provide educational resources for them to gain the technical skills, and more importantly skills of adaptability to reinvent their work. This helps minimize the chance of missed opportunities by limiting skills mismatch between talent and new demands created by new technologies. Upwork has played a significant role here by linking freelancers to resources such as Upwork Academy’s AI Education Library and Education Marketplace, thereby equipping them with the necessary tools and knowledge to adapt and thrive in an AI-present job market. This approach can help bridge the gap between low- and high-value work opportunities, ensuring a more equitable distribution of the advantages brought about by generative AI.
To estimate the causal impact of generative AI, we take a synthetic control approach in the spirit of Abadie, Diamond, and Hainmueller (2010). The synthetic control method allows us to construct a weighted combination of comparison units from available data to create a counterfactual scenario, simulating what would have happened in the absence of the intervention. We use this quasi-experimental method due to the infeasibility of conducting a controlled large-scale experiment. Additionally, we use Lasso regularization to credibly construct the donor pool that serves the basis of the counterfactuals and minimize the chance of overfitting the data.
Moreover, we supplement the analysis by scoring whether a sub-occupation is impacted or unaffected by generative AI. The scoring utilizes specific criteria: 1. Whether a certain share of job posts are tagged as AI contracts by the Upwork platform; 2. AI occupational exposure score, based on a study by Felten, Raj, and Seamans (2023), to tag these sub-occupations. We also use data smoothing techniques through three-month moving averages. We analyzed data collected on our platform from 2021 through Q3 2023. We specifically look at freelancer data across all 12 work categories on the platform for high-value contracts, defined as those with a contract of at least $1,000, and low-value contracts, consisting of those between $251 and under $500.
The main advantage of our approach is that it is a robust yet flexible way to identify the causal effects on not only the Upwork freelance market but also specific work categories. Additionally, we control for macroeconomic or aggregate shocks such as U.S. monetary policy in the pre-treatment period. However, we acknowledge the potential biases in identifying which sub-occupations are influenced by generative AI and the effects of external factors in the post-treatment period.
About the Upwork Research Institute
The Upwork Research Institute is committed to studying the fundamental shifts in the workforce and providing business leaders with the tools and insights they need to navigate the here and now while preparing their organization for the future. Using our proprietary platform data, global survey research, partnerships, and academic collaborations, we produce evidence-based insights to create the blueprint for the new way of work.
About Ted Liu
Dr. Ted Liu is Research Manager at Upwork, where he focuses on how work and skills evolve in relation to technological progress such as artificial intelligence. He received his PhD in economics from the University of California, Santa Cruz.
About Carina Deng
Carian Deng is the Lead Analyst in Strategic Analytics at Upwork, where she specializes in uncovering data insights through advanced statistical methodologies. She holds a Master's degree in Data Science from George Washington University.
About Kelly Monahan
Dr. Kelly Monahan is Managing Director of the Upwork Research Institute, leading our future of work research program. Her research has been recognized and published in both applied and academic journals, including MIT Sloan Management Review and the Journal of Strategic Management.
Autonomous Corporate Learning Platforms: Arriving Now, Powered by AIJosh Bersin 的文章通过人工智能驱动的自主平台介绍了企业学习的变革浪潮，标志着从传统学习系统到动态、个性化学习体验的重大转变。他重点介绍了 Sana、Docebo、Uplimit 和 Arist 等供应商的出现，它们利用人工智能动态生成和个性化内容，满足了企业培训不断变化的需求。Bersin 讨论了跟上多样化学习需求所面临的挑战，以及人工智能解决方案如何提供可扩展的高效方法来管理知识和提高学习效果，并预测了人工智能将从根本上改变教学设计和内容交付的未来。推荐给大家：
Thanks to Generative AI, we’re about to see the biggest revolution in corporate learning since the invention of the internet. And this new world, which will bring together personalization, knowledge management, and a delightful user experience, is long overdue.
I’ve been working in the corporate learning market since 1998, when the term “e-learning” was invented. And every innovation since that time has been an attempt to make training easier to build, easier to consume, and more personalized. Many of the innovations were well intentioned, but often they didn’t work as planned.
First came role based learning, then competency-driven training and career-driven programs. These worked great, but they couldn’t adapt fast enough. So people resorted to short video, YouTube-style platforms, and then user-authored content. We then added mobile tools, highly collaborative systems, MOOCs, and more recently Learning Experience Platforms. Now everyone is focused on skills-based training, and we’re trying to take all our content and organize it around a skills taxonomy.
Well I’m here to tell you all this is about to change. While none of these important innovations will go away, a new breed of AI-powered dynamic content systems is going to change everything. And as a long student of this space, I’d like to explain why. And in this conversation I will discuss four new vendors, each of which prove my point (Sana, Docebo, Uplimit, and Arist).
The Dynamic Content Problem: Instructional Design By Machine
Let’s start with the problem. Companies have thousands of topics, professional skills, technical skills, and business strategies to teach. Employees need to learn about tools, business strategies, how to do their job, and how to manage others. And every company’s corpus of knowledge is different.
Rolls Royce, a company now starting to use Galileo, has 120 years of engineering, technology, and manufacturing expertise embedded in its products, documentation, support systems, and people. How can the company possibly impart this expertise into new engineers? It’s a daunting problem.
Every company has this issue. When I worked at Exxon we had hundreds of manuals explaining how to design pumps, pressure vessels, and various refinery systems. Shell built a massive simulation to teach production engineers how to understand geology and drilling. Starbucks has to teach each barista how to make thousands of drinks. And even Uber drivers have to learn how to use their app, take care of customers, and stay safe. (They use Arist for this.)
All these challenges are fun to think about. Instructional designers and training managers create fascinating training programs that range from in-class sessions to long courses, simulations, job aids, and podcasts. But as hard as they try and as creative as they are, the “content problem” keeps growing.
Right now, for example, everyone is freaked out about AI skills, human-centered leadership, sustainability strategies, and cloud-based offerings. I’ve never seen a sales organization that does quite enough training, and you can multiply that by 100 when you think about customer service, repair operations, manufacturing, and internal operations.
While I always loved working with instructional designers earlier in my career, their work takes time and effort. Every special course, video, assessment, and learning path takes time and money to build. And once it’s built we want it to be “adaptive” to the learner. Many tools have tried to build adaptive learning (from Axonify to Cisco’s “reusable learning objects“) but the scale and utility of these innovations is limited.
What if we use AI and machine learning to simply build content on the fly? And let employees simply ask questions to find and create the learning experience they want? Well thanks to innovations from the vendors I mentioned above, this kind of personalized experience is available today. (Listen to my conversation with Joel Hellermark from Sana to hear more.)
What Is An Autonomous Learning Platform?
The best analogy I’ve come up with is the “five levels of autonomous driving.” We’re going from “no automation” to “driver assist” to “conditional automation” to “fully automated.” Let me suggest this is precisely what’s happening in corporate training.
If you look at the pace of AI announcements coming (custom GPTs, image and video generation, integrated search), you can see that this reality has now arrived.
How Does This Really Work
Now that I’ve had more than a year to tinker with AI and talk with dozens of vendors, the path is becoming clear. The new generation of learning platforms (and yes, this will eventually replace your LMS), can do many things we need:
First, they can dynamically index and injest content into an LLM, creating an “expert” or “tutor” to answer questions. Galileo, for example, now speaks in my own personal voice and can answer almost any question in HR I typically get in person. And it gives references, examples, and suggests follow-up questions. Companies can take courses, documents, and work rules and simply add them to the corpus.
Second, these systems can dynamically create courses, videos, quizzes, and simulations. Arist’s tool builds world-class instructional pathways from documents (try our free online course on Predictions 2024 for example) and probably eliminates 80% of the design time. Docebo Shape can take sales presentations and build an instructional simulation automatically, enabling sales people to practice and rehearse.
Third, they can give employees interactive tutors and coaches to learn. Uplimit’s new system, which is designed for technical training, automatically gives you an LLM-powered coach to step you through exercises, and it learns who you are and what kind of questions you need help with. No need to “find the instructor” when you get stuck.
Fourth, they can personalize content precisely for you. Sana’s platform, which Joel describes here, can not only dynamically generate content but by understanding your behavior, can actually give you a personalized version of any course you choose to take.
These systems are truly spectacular. The first time you see one it’s kind of shocking, but once you understand how they work you see a whole new world ahead.
Where Is This Going
While the market is young, I see four huge opportunities ahead.
First, companies can now take millions of hours of legacy content and “republish it” in a better form. All those old SCORM or video-based courses, exercises, and simulations can turn into intelligent tutors and knowledge management systems for employees. This won’t be a simple task but I guarantee it’s going to happen. Why would I want to ramble around in the LMS (or even LinkedIn Learning) to find the video, or information I need? I”d just like to ask a system like Galileo to answer a question, and let the platform answer the question and take me to the page or word in the video to watch.
Second, we can liberate instructional design. While there will always be a need for great designers, we can now democratize this process, enabling sales operations people, and other “non-designers” to build content and courses faster. Projects like video authoring and video journalism (which we do a lot in our academy) can be greatly accelerated. And soon we’ll have “generated VR” as well.
Third, we can finally integrate live learning with self-directed study. Every live event can be recorded and indexed in the LLM. A two hour webinar now becomes a discoverable learning object, and every minute of explanation can be found and used for learning. Our corpus, for example, includes hundreds of hours of in-depth interviews and case studies with HR leaders. All this information can be brought to life with a simple question.
Fourth, we can really simplify compliance training, operations training, product usage, and customer support. How many training programs are designed to teach someone “what not to do” or “how to avoid breaking something” or “how to assemble or operate” some machine? I’d suggest its millions of hours – and all this can now be embedded in AI, offered via chat (or voice), and turned loose on employees to help them quickly learn how to do their jobs.
Vendors Watch Out
This shift is about as disruptive as Tesla has been to the big three automakers. Old LMS and LXP systems are going to look clunkier than ever. Mobile learning won’t be a specialized space like it has been. And most of the ERP-delivered training systems are going to have to change.
Sana and Uplimit, for example, are both AI-architected systems. These platforms are not “LMSs with Gen AI added,” they are AI at the core. They’re likely to disrupt many traditional systems including Workday Learning, SuccessFactors, Cornerstone, and others.
Consider the content providers. Large players like LinkedIn Learning, Skillsoft, Coursera, and Udemy have the opportunity to rethink their entire strategy, and either put Gen AI on top of their solution or possibly start with a fresh approach. Smaller providers like us (and thousands of others) can take their corpus of knowledge and quickly make it come to life. (There will be a massive market of AI tools to help with this.)
I’m not saying this is easy. If you talk with vendors like Sana, Docebo, Arist, and Uplimit, you see that their AI platforms have to be highly tuned and optimized for the right user experience. This is not as simple as “dumping content into ChatGPT,” believe me.
But the writing is on the wall, Autonomous Learning is coming fast.
As someone who has lived in the L&D market for 25 years, I see this era as the most exciting, high-value time in two decades. I suggest you jump in and learn, we’ll be here to help you along the way.
About These Vendors
Sana (Sana Labs) is a Sweden-based AI company that focuses on transforming how organizations learn and access knowledge. The company provides an AI-based platform to help people manage information at work and use that data as a resource for e-learning within the organization. Sana Labs’ platform combines knowledge management, enterprise search, and e-learning to work together, allowing for the automatic organization of data across different apps used within an organization.
Docebo is a software as a service company that specializes in learning management systems (LMS). It was founded in 2005 and is known for its Docebo Learn LMS and other tools, including Docebo Shape, its AI development system. The company has integrated learning-specific artificial intelligence algorithms into its platform, powered by a combination of machine learning, deep learning, and natural language processing. The company went public in 2019 and is listed on the Toronto Stock Exchange and the Nasdaq Global Select Market.
Uplimit is an online learning platform that offers live group courses taught by top experts in the fields of AI, data, engineering, product, and business. The platform is known for its AI-powered teaching assistant and personalized learning approach, which includes real-time feedback, tailored learning plans, and support for learners. Uplimit’s courses cover technical and leadership topics and are designed to help individuals and organizations acquire the skills needed for the future.
Arist is a company that provides a text message learning platform, allowing Fortune 500 companies, governments, and nonprofits to rapidly teach and train employees entirely via text message. The platform is designed to deliver research-backed learning and nudges directly in messaging tools, making learning accessible and effective. Arist’s approach is inspired by Stanford research and aims to create hyper-engaging courses in minutes and enroll learners in seconds via SMS and WhatsApp, without the need for a laptop, LMS, or internet. The company has been recognized for its innovative and science-backed approach to microlearning and training delivery.