Harnessing AI & LLMs in Today's Data-Driven World

As the CTO of ipushpull, I had the privilege of participating in a panel at the recent FinJS event. The focus was on "Data, Cloud, and Analytics," particularly in the context of AI, including Large Language Models (LLMs). Reflecting on this experience, I want to delve into the real-world applications of AI, their integration into workflows, and the balance of hype versus substance in AI.

Real-World Examples of Using AI for Practical Problem Solving

At ipushpull we've embedded AI into our workflow framework, enabling us to use LLMs to extend our capability to map, transform, and parse data. We prioritise practical solutions, ensuring that the technology aids daily business operations, enhances efficiency, and reduces risks.

More specifically, we’re using a range of AI techniques to answer a question that many of our customers have asked us – ‘How can I interpret unstructured data and get it into my trading and pricing systems?’

The background is that our customers use ipushpull to automate their workflows with their counterparties. These are workflows between market participants, like banks and brokers, or sell sides and buy sides. In the OTC market, many of these workflows are not automated – they’re conducted through chat platforms, email, or by sending spreadsheets around.

We’ve built a flexible data and workflow integration platform that lets them ingest data from these communications platforms into their internal systems - like OMSs and EMSs. In some workflows, it’s possible to impose a fixed syntax or use mapping rules to automate this ingestion.

But many of our customers live in a world of unstructured data. Imagine you’re a broker receiving orders, or a trader at a buy-side receiving prices via chat. Every one of your counterparties uses a different format and there’s no way to impose a standard. To get this data into your own trading or pricing platforms you have to laboriously double-key it. And in a lot of cases, it never gets into a system at all – this is valuable market intelligence that’s going to waste.

Choosing AI: A Balanced View

The decision to implement Generative AI or AI approaches involves weighing their advantages against potential downsides. We've developed a model to evaluate where AI fits best, considering factors like performance, cost, flexibility, and data security.

In our experience, LLMs excel in specific tasks but may falter in others. The key is identifying the most suitable solution, even if it's not the most exciting or hyped one. For instance, when accuracy is paramount, but the data quality is questionable, we assess whether the overall pattern is more valuable than individual data accuracy.

Integration of LLMs into Workflows: An Essential Synergy

We’ve extended our workflow framework so that we can now call out to AI services or hosted models. We can apply a range of techniques like entity extraction with foundation models, or named entity recognition with fine-tuned models, to convert unstructured text to structured data.

By prompt engineering or fine-tuning models, we’ve built a process that ingests unstructured text and generates structured JSON which we then feed into the upstream systems.

The results we’ve achieved so far are remarkable. The models are capable of automating a high percentage of what was previously manual workflow.

However, it's critical to keep a human step in place so that users can approve or correct what the AI is suggesting. AI is very much a co-pilot, helping the user to remove tedious manual work from their processes.

Integration of AI into existing workflows is crucial. However, you can't have an AI strategy without a strong data strategy. At ipushpull, we've successfully integrated LLMs into our platform, demonstrating comprehensive front-to-back workflows where AI plays a significant role.

Prompt Engineering: A New Skill

Working with AI provides new challenges for developers and data scientists. For example, Prompt engineering – providing additional context and instructions to models – has become an essential skill. It's a quick way to develop initial solutions and iterate on them, but it's a new world for developers because results can be non-deterministic and the output of models is not explainable. If you’ve developed a traditional parser using regexes you can explain why it works and debug it when it fails. In comparison, changes to prompts can unpredictably affect previous results, requiring a careful approach and thorough regression testing. The key to success is building and maintaining large training and testing sets. So when you fine-tune a model or update a prompt you can immediately regression test it.

Fine-tuning Open Source Models: An Untapped Resource

Large providers of general-purpose models dominate the LLM landscape. But often these general-purpose models are not the best option for supporting a specific workflow. As an alternative, specialised open-source models offer fine-tuning, customisation, performance and cost benefits, plus intellectual property retention. We assist clients in leveraging these models and integrating them into their existing platforms and workflows.

The Future: Beyond Hype

The excitement surrounding AI is undeniable, but at ipushpull, we focus on the tangible value it brings to our customers. As we progress, we anticipate LLMs and other AI models becoming quicker and more flexible, making them even more integral to our data-driven solutions.

The integration of AI into our workflows is not just about the technology itself but how it harmonically blends with existing data strategies and business processes. At ipushpull, we're poised to lead this journey, embracing the transformative power of AI while grounding our approach in practical, real-world applications. 

We're thinking big, but starting small. Successful projects require carefully navigating the route between hype, scepticism, and fear to ensure we get everyone on board for the journey and show them how AI can be a co-pilot to streamline their work. If you found this article interesting, why not contact us to see how we can help you on your AI journey?