Matthew Cheung: Hi, this is Matthew Cheung and I am CEO of ipushpull. This is the What is your chat strategy podcast. Today, we're very happy to be joined with Andreas Hoff, who leads product strategy and execution at Trayport. And Trayport is the primary network and data platform for the European wholesale and energy markets.

Matthew Cheung: So thanks for joining us today, Andreas. Thanks for having me. And so we're going to discuss ideas and your thoughts around. Chats, chatbots, large language models, and how you see things evolving in this super fast moving space. So Trayport is obviously moved used in energy and commodities markets.

Matthew Cheung: But what chat platforms do you generally see your customers using?

Andreas Hoff: Yeah, so we service traders, brokers, and exchangers. They obviously need to interact with each other for the the trading activities that they're doing. So I see basically two main categories. There are the industry solutions, so out of the financial and commodities trading industry, you have chat platforms from Bloomberg, Refinitiv, IceChat, Symphony and they are really aimed at that specific use case of.

Andreas Hoff: Interactions between trading firms and counterparties, but you also see a lot of usage on your traditional consumer type applications. So just because everyone has access to them and all you need is the other person's phone number. It's very convenient for people to get started just chatting to each other on WhatsApp or WeChat in Asia.

Andreas Hoff: Some people use telegram. And yeah, I think it's, it's a big mix of multiple systems that the, the traders and brokers are using today. And if you want access to as many people as possible, you probably need to use multiple of these.

Matthew Cheung: So if I'm a broker, for example, then I've got all those different chat platforms you went through.

Matthew Cheung: Maybe one broker might have all of those chat platforms or a subset of them. But is, is there very much like a proliferation of chat and is that, do you think that's a problem for brokers or traders who've got limited screen real estate anyway? How, how can they be able to, to, to give the right amount of that screen real estate to the right chats or do they have to just have them all up with lots of notifications blasting away?

Andreas Hoff: Yeah, I think there's some of that. What you often see is that de facto defaults established in certain subsections of the market. So you may have one particular platform that's very common. In in one asset class and then a different asset class. It's a different platform. But yeah, especially for brokers, they want to be accessible and available to as many counterparties as possible.

Andreas Hoff: So they will often have multiple of these platforms up and running on their desktops, and they have to switch between them. And yeah, the challenges is not just switching between applications, but even within a single application, you sometimes have Dozens of chat windows open and I imagine it's, it's a bit of a challenge for, for any broker or trader to just remember where all my chat windows are and who I still need to reply to.

Matthew Cheung: And historically, commodities markets have been very chat driven, voice driven, and not quite as electronified as some of the financial markets, you know, in equities and other areas like that, because we've seen, obviously, the rise of chat GPT a year ago now, everyone's starting to have a look at how, how can I think about their own chat strategies?

Matthew Cheung: How can they incorporate something like the power of chat GPT into what they're doing? Have you seen a lot more interest around Let Automating these voice and chat workflows, you know, using AI and, and at large language models and so on. What, what have you seen with your customers and what are you seeing within your own company?

Andreas Hoff: Yeah, we see a pretty wide spread of what, what people are trying and what they'd like to do. I think on the, on the most advanced. Scale. You have people thinking, Oh, we can just have chat GPT do the whole trading for us, right? It can just be a chat bot and it. Understands what everyone wants and just replies automatically.

Andreas Hoff: I think we are still a fair way off that being being a realistic use case. So I think an important step to adopting some of these technologies is just understanding what they're good at and what their limitations are. What we find that can be really good at is extracting information. So if you have natural language, like a chat history.

Andreas Hoff: LLMs and Chechi BT style systems can be really good at identifying what the conversation was about and extracting information like what price or quantity, which contract they were talking about. If they're instructed well, I think these types of systems are already pretty good at that. They are not.

Andreas Hoff: 100 percent reliable. So I think it's something that we find a lot of our customers say is we're looking for solutions that have a human in the loop. So you don't want a system that is working off of probabilities and sometimes goes down the wrong path. You don't want that to be fully autonomous in your trading life cycle at this point.

Andreas Hoff: So what you want rather is something that still has a step where a human decides, well, that information that you extracted, do I agree with your assessment of that? So it can speed up that process a lot, but you still have the human to, to just do the quality. Check at the end and equally on the on the automation side, if the system does a recommendation of how you should respond to this quote request with these quotes, you still want to step in the middle where the human can.

Andreas Hoff: Decide, is that really what the person asked for and all those prices, the ones that I should be showing

Matthew Cheung: and also the human in a lot of cases is a, is a regulated person. So it also makes that process of rolling out these type of technologies slightly easier than if you're handing everything over to a black box, which then it's difficult then to prove to a regulator.

Matthew Cheung: If you have to justify a decision that it may have made. Absolutely. So, so in terms of, I mean, you're the chief products officer at Trayport , so you've seen all of these technologies, and I'm sure you've played around it in your personal time like everyone has. But in terms of what you're doing inside Trayport and utilizing ai, how, how did you go about thinking about an AI strategy and having the, the, the data that you can use and having the.

Matthew Cheung: the integrations to be able to plug things across chat into trade port and so on. How are you thinking about that at the moment? And are you, are you progressing those ideas into to get things in front of customers?

Andreas Hoff: Yes, we do. The way we approach it is always with the customer mindset and customer problem first, right?

Andreas Hoff: So we, we don't look at it as we need to get a new technology. Out into the world. But we look at it in terms of what our clients problems that we can solve with these new capabilities. So the way we approach it, we do a lot of prototyping. We do a lot of Exploration. So we try out different models.

Andreas Hoff: We've, we've probably used the GPT model from open a I the most in our explorations. And then we work really closely with a few select customers who are keen to to explore that with us. So that way we can make sure that Every iteration we do, every idea we bring in has that customer feedback on it.

Andreas Hoff: So we go, we don't go down the wrong path imagining what people's problems are, but we get the direct feedback from customers of what they find valuable and problems that they see. So part of, part of the whole human in the loop thing, that's direct learning from customers where basically.

Andreas Hoff: Everyone we speak to wants that and needs that.

Matthew Cheung: Why did you choose to use OpenAI? Was that because it was easy to access through Azure? Or just because it was the main model that people are playing with?

Andreas Hoff: It's a combination of both, right? One benefit that you get is that you probably have used chat GPT.

Andreas Hoff: So you have at least the starting point of understanding what, what the model is capable off and where it falls down. So that is really helpful to get started because you can already Kind of tell which use cases you you want to try it with. We do use it through Microsoft Azure. We do use it through Microsoft Azure.

Andreas Hoff: So that ensures that we can use it in a private instance. So the data that we put in is not being used for any sort of. Training is not leaking to any third parties. That's really important to us and our customers. And we're also very, very careful with the data we put in for testing. So rather than putting customer data in, we generate some example data.

Andreas Hoff: Of our own because we need customers to agree to any, any of such use cases before we put their real information in.

Matthew Cheung: And how do you see the, the use of, so obviously there's people like yourselves bringing out these new, new solutions to old problems. Whereas a lot of manual processes flying around and things like a large language model can extract and capture price information or trade information.

Matthew Cheung: And help that integrate or help that process of integrating into your existing platform. For example, how, how do you see the use cases of large language models as we go forward through 2024? Because there's, there's some, we're talking very specifically about chat here, but the other side is obviously data and summarizing data, querying data very quickly.

Matthew Cheung: What how do you see kind of the industry looking at things as we go forward another year another year ahead?

Andreas Hoff: Yeah, I think these llms will increasingly be used to capture and store the the structured information from your chat messages Right. Sometimes it still takes a brilliant trader to remember that that exact chat where someone was requesting a price on a, on a particular contract a week and a half ago.

Andreas Hoff: And you need to keep that in your mind at, at the moment with the, with the current chat systems, right? What you're going to see is that you can just ask the system, Hey, this is the contract I'm now looking at a counterparty on who was talking about this. Or who was talking about related contracts or who, who was generally in the market to, to sell that assets, the asset.

Andreas Hoff: So extracting that information, making it searchable doing some analytics on it as well. So that you can get an idea of how overall maybe buyers and sellers come in and drop out over a period of time. So that's really. a big use case that in the next few years, I think we'll really take off making that information more accessible, more searchable.

Andreas Hoff: The other thing is around automation. So I think we'll see more sort of recommender solutions where in the same way that your, your internal chat application may give you a few Options to pick from to auto respond. I think we'll see much smarter versions of that that can really be tailored to the financial use case where the system understands the context of what you're talking about in that chat, and it can pop up suggestions that are really And that can include information from other parts of the system.

Andreas Hoff: So we have the trading platform on, on trade port. Of course, people already have a bit of functionality there. They can right click on a grid and. Copy to instant messenger. So that makes it easy for them to just get the price as it is on the system right now and and put it into messaging platforms.

Andreas Hoff: There's a lot more information, a lot more automation that we can offer to our customers by just understanding what are those chat conversations about. And what are suitable things to ingest in that?

Matthew Cheung: And are you agnostic as to what chat your customers are using?

Andreas Hoff: We are agnostic to the chat that customers are using.

Andreas Hoff: I think. What you may see is that we will look for solutions that we can embed into, into the trade port screen, because we believe that the closer the integration is between your, your electronic trading and your chat platform, the better the user experience is going to be.

Matthew Cheung: Yeah, definitely. It's all about less, less clicks and copy pasting, which makes the whole process more efficient.

Matthew Cheung: Great. Okay. Andrea Soff. Thank you very much.

Andreas Hoff: Thank you, Matt.