Enterprises struggle to define generative AI strategies despite $14 billion investment

Enterprises Pour Billions into AI but Struggle to Define Its Role

Artificial intelligence (AI) is having a blockbuster year, with enterprise investments skyrocketing to nearly $14 billion in 2024. Yet, despite this massive spending spree, many companies are still scratching their heads over how to effectively implement the technology. A recent survey conducted by venture capital firm Menlo Ventures reveals that more than a third of businesses lack a clear vision for integrating generative AI into their operations.

“More than a third of our survey respondents do not have a clear vision for how generative AI will be implemented across their organizations,” wrote Menlo Ventures partners Tim Tully and Joff Redfern, along with investor Derek Xiao, in their report. Interestingly, the report itself was compiled with the help of Anthropic’s Claude Sonnet 3.5, a large language model (LLM).

The full report, titled 2024: The State of Generative AI in the Enterprise, is available on the Menlo Ventures website. It is based on responses from 600 IT decision-makers surveyed in September and October 2023. The findings paint a picture of an industry in flux, with optimism running high but strategic clarity still in short supply.

Generative AI: A Transformation in Its Early Stages

The authors of the report suggest that the uncertainty surrounding generative AI (Gen AI) is a sign that the technology is still in its infancy. “We’re still in the early stages of a large-scale transformation,” they noted. Despite this, the numbers tell a story of rapid growth and enthusiasm. Spending on generative AI tools and infrastructure has surged from $2.3 billion in 2023 to a staggering $13.8 billion in 2024. This includes investments in foundation models, model training and deployment, AI-specific data infrastructure, and new generative AI applications.

“This spike in spending reflects a wave of organizational optimism,” the report states. In fact, 72% of decision-makers anticipate broader adoption of generative AI tools in the near future. The largest chunk of this spending—$6.8 billion—has gone toward foundation models, the LLMs developed by companies like Anthropic and OpenAI. Meanwhile, spending on data and infrastructure remains the smallest category at $400 million.

AI Applications: The Fastest-Growing Segment

While foundation models dominate enterprise AI spending, the application layer is now growing at a faster pace. Spending on AI applications has increased eightfold, reaching $4.6 billion in 2024. This category includes three subcategories:

• Vertical AI: Industry-specific applications.
• Departmental AI: Tools tailored for specific business functions.
• Horizontal AI: Broadly applicable tools that cut across industries.

“The application category is heating up,” the researchers wrote. Companies are leveraging these tools to optimize workflows across various sectors, paving the way for broader innovation. Among the most prominent use cases are code generation tools like Microsoft’s GitHub Copilot, which is on track to generate $300 million in annual revenue. Other popular applications include support chatbots, enterprise search and retrieval systems, and automatically generated meeting summaries.

Anthropic Gains Ground Against OpenAI

Menlo Ventures has a vested interest in the AI space, backing several startups, including Anthropic and vector database maker Pinecone. According to the report, Anthropic is gaining traction in the enterprise market, with its Claude model winning converts from OpenAI’s GPT-4.

“Among closed-source models, OpenAI’s early mover advantage has eroded somewhat, with enterprise market share dropping from 50% to 34%,” the report states. “The primary beneficiary has been Anthropic, which doubled its enterprise presence from 12% to 24% as some enterprises switched from GPT-4 to Claude 3.5 Sonnet when the new model became state-of-the-art.”

The Modern AI Stack: Building the Future

One of the most forward-looking sections of the report focuses on the “Modern AI Stack,” a framework for the layers of infrastructure technology used to build AI applications. The researchers note that enterprises are beginning to coalesce around these core building blocks, which include:

• Foundation Models: The base layer of AI systems.
• Data Services: Tools like Pinecone that manage and organize data.
• Software Development Frameworks: Platforms like LangChain for orchestrating AI agents.
• Integration Tools: Solutions like Composio that connect AI systems with existing workflows.

This layered approach is helping companies streamline the development and deployment of AI applications, setting the stage for more sophisticated use cases in the future.

Predictions for 2024: Disruption and Talent Shortages

The report concludes with three bold predictions for the year ahead:

1. AI Agents Will Disrupt Enterprise Software: Platforms like Clay and Forge are expected to tackle complex, multi-step tasks, challenging traditional enterprise software solutions.
2. Established Firms Will Face AI-Native Competitors: Companies like Cognizant, UiPath, Salesforce, and Autodesk could see their markets disrupted by AI-native challengers.
3. A Talent Drought Will Hit the Industry: As AI adoption accelerates, the demand for skilled professionals will outpace supply, leading to soaring salaries for AI architects and data scientists.

“Brace for soaring competition and 2-3x salary premiums for already well-paid AI-skilled enterprise architects,” the researchers warn.

As enterprises continue to navigate the complexities of generative AI, one thing is clear: the technology is reshaping industries at an unprecedented pace. While challenges remain, the opportunities for innovation and growth are immense.