Podcast Review: Insights from SAP’s VP of AI Product Engineering, Mayank Sharma

In a rapidly evolving technological landscape, SAP has emerged as a key player, pushing boundaries with its AI-driven strategies and products.

In a recent episode of AI Business Asia, host Leo Jang had an engaging discussion with Mayank Sharma, SAP’s Vice President of AI Product Engineering.

Sharma, who also heads SAP’s AI Services and Accelerator, offered a fascinating deep dive into SAP’s AI journey, its transformation over the years, and how it is shaping the future of enterprise solutions.

Here’s a detailed recap of the episode, packed with actionable insights and exciting developments from the world of AI.

SAP’s Evolution: From ERP Giant to AI Trailblazer

For nearly 50 years, SAP has been synonymous with enterprise software solutions, particularly in ERP and CRM systems.

Known for its modular, customizable—and often complex—solutions, SAP’s legacy is deeply entrenched in facilitating enterprise operations.

However, the company’s journey in AI started in earnest in 2015, well before the explosion of generative AI and Transformer-based models like ChatGPT.

Sharma described this early phase as a time of exploration, focusing primarily on classical machine learning and deep learning models.

These required significant effort in data integration, preparation, and training pipelines. The process, while groundbreaking, faced hurdles, including prolonged timelines from ideation to production and high entry barriers for customers with limited data infrastructure.

The Generative AI Revolution

The advent of generative AI models, especially large language models (LLMs), has been transformative for SAP and its customers.

Sharma explained that generative AI has significantly reduced the time-to-market for AI solutions, thanks to its ability to deliver results with minimal training data via zero-shot and few-shot learning.

This shift has also lowered the barriers to entry for enterprises looking to adopt AI, making it easier for businesses to integrate these capabilities into their workflows.

Generative AI is not just about efficiency—it’s opening new doors for innovation. According to Sharma, SAP has begun leveraging AI for content generation, agent-based workflows, and revolutionary user experiences through AI co-pilots like Joule.

These advancements are reshaping how businesses interact with SAP systems, offering more intuitive, automated, and intelligent solutions.

Joule: SAP’s AI Co-Pilot

One of SAP’s flagship AI innovations is Joule, an AI co-pilot designed to enhance user experiences across enterprise systems. Joule integrates seamlessly with SAP’s ecosystem, offering capabilities such as:

• Information Extraction: Simplifying complex navigation by delivering precise data upon request.
• Transactional Assistance: Automating multi-step processes like sales order management by identifying and resolving blockers.
• Analytical Insights: Providing actionable recommendations and contextual data to aid decision-making.

Sharma highlighted a real-world example: a blocked sales order process. Traditionally, resolving such issues required navigating multiple menus and substeps.

With Joule, users can simply ask the AI to identify and resolve blockers, completing the process in minutes. Joule’s integration extends to voice commands, mobile platforms, and desktop applications, ensuring accessibility and convenience.

Addressing Data Privacy and Security

As enterprises increasingly bring their data to the cloud to leverage generative AI, concerns around data privacy and security are paramount. SAP has built robust mechanisms into its AI offerings to ensure data protection.

Joule, for example, incorporates user-specific authorization, ensuring that sensitive information is accessible only to authorized personnel. This approach maintains trust and compliance while maximizing AI’s potential.

A Flexible Approach to LLMs

SAP’s AI strategy embraces both open-source and proprietary LLMs. Through its Generative AI Hub, SAP collaborates with leading providers like Meta, Microsoft, Google, and IBM, offering customers access to the best-fit models for their unique needs. This hybrid approach allows SAP to evaluate and benchmark models continuously, ensuring that its solutions remain cutting-edge.

Sharma emphasized the importance of flexibility in LLM adoption. “We cannot lock ourselves into one model or provider,” he said, noting that SAP’s AI landscape architecture is designed to evolve alongside advancements in LLM capabilities. This adaptability enables SAP to deliver consistent value to customers while staying ahead in a fast-paced industry.

Agent-Based Workflows: A Glimpse into the Future

A key focus area for SAP is agent-based workflows, which leverage specialized AI agents to perform tasks autonomously. These agents excel in specific domains—such as finance or HR—and collaborate seamlessly to execute complex, cross-functional processes. Sharma described how this innovation is transforming business operations by reducing manual intervention and enhancing efficiency.

For instance, a transaction requiring data from both HR and finance systems can now be managed by AI agents that interact autonomously, delivering results without user intervention. While SAP is still in the process of refining these workflows, the potential for end-to-end automation is immense.

Empowering Partners and Startups Through SAP’s Ecosystem

SAP’s commitment to innovation extends beyond its own offerings. Through initiatives like SAP Ventures and SAP App Store, the company collaborates with startups and partners to co-create complementary applications. These solutions, built on SAP’s Business Technology Platform, benefit from seamless integration, robust authorization, and data management capabilities.

Sharma highlighted the growing role of generative AI in this ecosystem, with over 100 partners developing AI applications via SAP’s Generative AI Hub. These apps are published on the SAP App Store, making them easily accessible to customers and driving a win-win scenario for startups, partners, and end-users alike.

Challenges and Opportunities in Asia’s AI Landscape

Discussing AI innovation in Asia, Sharma noted the region’s unique challenges and opportunities. While diversity in culture and business dynamics creates a rich environment for tailored solutions, the maturity of AI adoption remains uneven. Global players dominate the space, but the emergence of regional innovators is a promising trend.

SAP is actively engaging with local startups and exploring opportunities to foster AI innovation tailored to Asia’s specific needs. The company’s generative AI initiatives aim to empower enterprises across the region, helping them unlock new efficiencies and business models.

Building a Sustainable AI Strategy

Sharma offered valuable advice for organizations crafting their AI strategies:

1. Prioritize a Strong Data Foundation: Harmonized and anonymized data is critical for effective AI adoption.
2. Invest in Flexibility: AI architectures must evolve to accommodate rapidly changing LLM landscapes.
3. Ensure Privacy and Security: Robust mechanisms to protect data and manage access are non-negotiable.
4. Embrace Automation: AI-driven workflows should enhance user experiences and streamline business processes.

He stressed that a sound AI strategy is built on the foundations of digital transformation and data strategy, ensuring that enterprises can adapt and thrive in an AI-first world.

The Road Ahead: AI-First Solutions

Looking to the future, Sharma is optimistic about the rapid advancements in AI. He envisions a world where AI-first strategies dominate software development, enabling businesses to achieve unprecedented levels of efficiency and innovation. From agentic workflows to enhanced user experiences, SAP is poised to lead this transformation.

“I feel fortunate to be part of this exciting journey,” Sharma said. “Every day brings new opportunities to innovate and redefine what’s possible.”

SAP’s journey in AI is a testament to the transformative power of technology. By embracing generative AI, fostering partnerships, and prioritizing customer-centric innovation, the company is not only redefining its legacy but also shaping the future of enterprise solutions.

For businesses exploring AI adoption, SAP’s strategy offers a blueprint for success: focus on flexibility, prioritize user-centric design, and never stop evolving.