MuleSoft unveils composability solution for gen AI | CIO

Oct 31, 2024

Salesforce subsidiary MuleSoft announced a new AI-powered composability solution on Tuesday aimed at helping organizations build discrete AI services that form the building blocks of more complex systems and applications.

“We believe the world of AI is really the world of APIs,” said Param Kahlon, Salesforce EVP and GM of Automation and Integration. “If you look at how people are accessing AI in the enterprise, it really is the ability to call a model.”

In the case of gen AI, that call may be to an LLM hosted within the enterprise, but it also might be to a foundational model built by a third party.

“Whether you’re using an LLM that’s within the company or using an outside model or federating a request across multiple LLMs through LangChain, it’s all involving API calls,” Kahlon says. “All these API calls need to be managed and governed.”

Kahlon said MuleSoft’s goal is to create a platform that helps customers integrate AI, especially gen AI, with their business processes.

“For example, we are looking at how, if you’re making LLM calls outside of your organization, how can our API management tools be used to manage those API calls,” Kahlon said. “We’re also looking at how our APIs can be used as actions by copilot conversational agents in the enterprise. Instead of figuring out how to call an API to make an action in the back office, what if you can train the agent to understand that API and make natural language inferences to execute on this API to give the customer credit in the back office or escalate an order?”

The MuleSoft composability solution consists of three pillars: the Anypoint Platform (used to define, design, build, and deploy APIs), API management (to manage the deployment of APIs through their lifecycle, whether Mule APIs built with Anypoint or APIs built using another technology and architecture), and automation (including MuleSoft RPA and MuleSoft Intelligent Document Processing, or IDF).

While some of these components are already part of MuleSoft’s portfolio, the company announced new features, including support for AsyncAPI to facilitate the adoption of event-driven architectures (EDAs).

The AsyncAPI support in Anypoint is currently in open beta, though Kahlon said the company will make it generally available later this year. AsyncAPI support will enable systems to add real-time communication to systems or processes that contain fluctuating data sets, like predictive maintenance, dynamic pricing, or fraud detection.

For example, a bank might use AI models to analyze transactional data and user behavior patterns for fraud detection. With the composability solution, the bank could turn that model into a service that can be called by various applications.

“A fraud detection experience is powered through a set of streaming data that might be built into a customer service application,” Kahlon said.

“At the point of taking the call, you might be passing it to an AI system that’s detecting the call and understanding the propensity for it to be fraud based on past interactions. All that could be built outside of your core customer service application. If you build it as an independent application, you might, over time, replace the customer experience that you’re delivering without having to change the fraud detection algorithm.”

Kahlon noted that security and governance are key components of the composability solution. When applications make API calls to LLMs and other models outside the corporate boundary, it’s essential that valuable data be encrypted and/or masked.

The company’s API gateways, Anypoint Flex Gateway, and Mule Gateway can act as LLM gateways with custom-made policies to secure and manage APIs. For example, a financial institution could use an API gateway to create a custom policy that checks for sensitive customer information before data is shared with a third-party LLM.

More MuleSoft news: