Dan Eccleston: I'm delighted to be joined by Peyman Mestchian, Director of Research and Professor at University College Institute of Finance and Technology. Peyman's been both founder and advisor to many fintech firms, large and small, over the years and has a wealth of experience in AI. So we'll hopefully be able to give some great insights into what's happening now Peyman.

Peyman Mestchian: Thank you. Thanks for having me. Yeah. Nice to talk to you. Thank you. So to kick off a question on AI evolution, maybe a little bit of history and where we are now. So you've been involved around AI for many years and large language models seems to have. suddenly became more accessible and useful in the last couple of years, which you know, you must have been noticing and that's not just in financial service, but across the board.

Peyman Mestchian: Do you think general awareness has suddenly been stimulated by the likes of chat GPT? Has that, has that been a major catalyst for that? 'cause it's not like this stuff just appeared out of nowhere, or what are the other factors that, for the certain interest? So yeah, the short answer is yes. So, you know LMS and chat, C p t their adoption know, has been exponential.

Peyman Mestchian: But I think they are, they are not the reason they're not the, they're not the cause for increased use of AI. I mean, AI is as a discipline is actually not a lot of people know, but it's probably about 50 years old. It's been around for decades. And it's gone through many many cycles of, you know, trial and error and different techniques and so on.

Peyman Mestchian: I think what's really brought it to the fore in, in recent years is A number of different factors coming together at the same time. I think one is around the availability of data. So modern day businesses and society as a whole, we are generating so much data and the volume and complexity of data is increasing exponentially.

Peyman Mestchian: So, and it's available. So, you know, All the way from the internet, you know, social media internal, external to the organization, you know, we're surrounded by data and in all shapes and forms, structured, unstructured, semi structured and so on. So there's a mass explosion of data. So that's one vector. The other vector is actually to do with hardware.

Peyman Mestchian: So processing, processing power, the, the in terms of the, you know, GPUs and, and, and. Both in terms of speed and cost hardware performance is, you know, as, you know, at the highest level it's been and it's, you know, improving the speed and performance of hardware is improving. I think, I think statistics say it's like every 18 months it's doubling.

Peyman Mestchian: So that's, that's a factor and it's, it's more generally available, whereas, you know, maybe a few decades ago, it was only available to the very large organizations with deep pockets or very large labs or sort of research institutes at universities. It's now just generally available. So the hardware, the data coming together, and then yes, there's some, you know, there's improvement in actual AI techniques in terms of models, you know, neural networks and the way we, we use that data and the way we use that hardware to train AI models.

Peyman Mestchian: So those are, those three forces coming together over the last few years have resulted. in fantastic innovations such as large, large language models, which then you are generating text and, and, and creating also productivity benefits. I guess, interestingly chat GPT was able to be given away for general consumption for people to try it because it was suddenly cost effective or even cost possible to be done at that time because of all those, the confluence of all those things about that particular moment.

Peyman Mestchian: Yeah, but look, the, the start things with free free models and then, you know, find other ways to monetize it is a, is a tried and tested model by our big, big technology friends with better base, you know, free, free Facebook or free you know Google searches are free. So there's, you know, it's, it's a, it's a, You know, if something is so powerful and you put it in the hands of people for free, they understand the benefit and the power of it.

Peyman Mestchian: And then once you get the nonlinear growth in adoption, then the monetization opportunities come down the road. So I mean, you have a bit open source. I mean, yeah. It's called OpenAI for a reason. It's, you know, it's basically it's open source. So that's the, that's the good, well established, tried and tested open source model which I think is the way in the future.

Peyman Mestchian: And now, as you've seen, the other big players, Microsoft, Amazon, you know, are all following, following suit. I mean, even even, you know, the earlier versions of this around IBM Watson, there were three versions of IBM Watson also available for downloading that, you know not as mature, mature as this type thing, but in terms of natural language processing, natural language understanding yeah using the free, free, initial free based model to, to get adoption is, is is the way forward, yeah.

Peyman Mestchian: Yeah, let's come back to the financial model a little bit later because I've got some good questions on that. Hopefully so, so one of the powers of large language models, obviously interpreting less structured data and specifically natural language. Right. That could be voice. It could be email more and more.

Peyman Mestchian: It's, it's chat. My company pushpool deal deal with all of those predominantly there's a lot of interest in chat at the moment. As, as, you know, in trading and risk. Landscape a lot forwards on chat all the time. What one of our challenge, one of the challenges we're seeing is how we can understand how accurate the outputs are in relation to how accurate they need to be from a trading perspective.

Peyman Mestchian: And also from a risk perspective, and I'm wondering if you see any of the organizations you're involved with dealing with the same problems and how you think those. Problems are being dealt with part or partly from a, any sort of systematic way of, of understanding how accurate things do need, need to be and how to measure that.

Peyman Mestchian: Or, or is it still at a stage of very kind of suck it and see, have a go. And You know, that kind of a more early development stage particularly trading and risk are the key ones because you need stuff to be accurate, right? Well, look, I mean, maybe step back and think about what live language models are.

Peyman Mestchian: You know, they're not search engines that there are large language models of fundamentally statistical models that are trained using data. So it could be unstructured data structured data. So statistical models, which have got probabilities of certainty to your point around, you know, accuracy and so on.

Peyman Mestchian: So of course, the richer, the more complete, the more comprehensive, the higher quality the data is. That's being used to train these models, then the better the output depending on the domain of application. Sometimes data is lacking. So I mean, for example, like use using, you know, extracting signals from news on structured data.

Peyman Mestchian: To, to suggest signals for generating alpha in the trading environment in your world of, you know, capital markets, that's been around for some time, you know, in, you know, hedge funds and, you know, algorithmic trading and so on. And of course, the more you train it, the better it gets. So, you know, it's that's been around.

Peyman Mestchian: So in large language models. What's happened in the last year or two is more around generating text rather than understanding text. The text understanding, you know, like extract, extracting an entity, be it a company, an equity company from, from news and then understanding adverse media, negative news about it, or positive news, and then scoring the sentiment, which then feeds into some sort of risk, risk model for trading decisions and so on.

Peyman Mestchian: That's been around. Of course, this type of neural, neural net based training is, is constantly enhancing it. So That's fine. And, and I think, you know, we'll carry on getting better at that. I think the point you're making around accuracy, though, is when we move into the world of regulated activities, which, you know, trading investment decisions, you know you know, investment advice, you know, using, using AI for investment advice there, as you know, in the regulated environments, you need model governance and model transparency and model explainability.

Peyman Mestchian: So that's not just explaining the model itself. When data is being, when models are being trained at such large scales with such large data sets, it's very difficult to explain why a piece of advice or a recommendation or a signal is coming out of the model. The regulators actually require you know, a clear auditable trail of understanding these recommendations because is there bias in the data?

Peyman Mestchian: Is the data quality, you know, you know, quality controlled? And so, so there's a data quality control aspect to it. There's a model governance aspect to it. And I think that side, that aspect of it around risk and compliance we are in early stages, but it's going to be a massive area. Massive area and you know, there will be there will be wrong decisions made because of, you know inaccurate inputs, which resulting in inaccurate outputs.

Peyman Mestchian: And, and, and, you know it can have multi billion dollar implications that's going to happen. So all we can do is to enhance the governance and control mechanisms around this and make sure there's sufficient human in the loop and subject matter expertise. Cheers. Before decisions are actually, you know, somebody presses a button.

Peyman Mestchian: And I think these are, these are emerging fields. Yeah, and it seems for, certainly for many, trading at risk. Workflows that the human in the loop, at least at the initial stages is absolutely critical and then, and then it can be dialed down to a certain extent by, by putting further rules and deviations and filters around it.

Peyman Mestchian: But I totally agree that the human in the loop is really key. Yeah, there's a spectrum that some of the work we do at UCL and I've seen, you know, research from other institutions and some of the companies that I work with is there is an emerging field as a subset of AI, which is called decision intelligence.

Peyman Mestchian: Which actually is a for those of you know, those of the from your audience. Those who are familiar with this sort of world of enterprise software and so on. You know, decision intelligence is sort of half artificial intelligence, half business intelligence around, you know, you know, dashboards and visualization of information.

Peyman Mestchian: Decision intelligence sits in the middle and there's a spectrum of sophistication from just decision support. So so basically the system provides a set of recommendations. And But ultimately, it's the human who makes the decision all the way to the other side of the spectrum, where it's actually the machine making the decision.

Peyman Mestchian: There's no human, it's an automated, fully automated decision process, decision scoring. And, you know, depending on the domain, you know, we, we already have like credit decisioning in the financial sector. And, and there are, there are systems like that do medical diagnosis is the machine actually recommends this is, this is what you've got.

Peyman Mestchian: And the doctor really comes right at the end to just, you know, verify it. Or not and and so depending on the domain and again availability of data and accuracy of data, the level of automation and how the man machine allocation of functionality depends on availability of data and also the nature of the task.

Peyman Mestchian: But of course, the more safety critical activities such as medical diagnosis or, you know, making decisions about, I don't know, earthquake signals or air traffic control and so on. Thank you. That's where, you know we're not at a stage where we can just give everything to the machine. So yeah, yeah, presumably there's a certain amount of AI driving cars already and probably flying planes that we don't really want to think about it.

Peyman Mestchian: Right. So it's interesting that that precedent set in those really critical areas. And it's probably probably behind to a certain extent, aren't we in there's certainly in, in capital markets and regulatory side in, in, you know, belts and braces of yeah, I mean, your, your air traffic control and, and, and autopilot type stuff is a really good example because the safety control systems around the machine decision or machine recommendations are incredibly Yeah.

Peyman Mestchian: Incredibly sophisticated and well tested and the human in the loop is built into the system to make sure, make sure that the responsibility and accountability is not given to the machine, it's given to the human. So yeah, so there's a whole area of sort of AI governance, which I think is going to be one of the fastest growing sort of Consequences of this adoption.

Peyman Mestchian: Yeah, yeah, yeah. Just, just back on the building an AI model or a large language model and training it. I don't know what your thoughts and experience of obviously it's, it's become much more easy, more recently to be able to license. And utilize a third party model. Is there a certain point that you've seen where people say I mean, you know, maybe you say you know, 10 years ago, you had to build your own, right?

Peyman Mestchian: Because there just wasn't much useful out there. I don't know. But now, obviously, there's a, there's a number of competing services. All vying to get people hooked in you know, from Google now, like Amazon and Microsoft, so obviously, you know, all the famous so what do you think about the, so some of them are open source.

Peyman Mestchian: They're trying to get people to license. Is there an actual cutoff point and what type of business should be building their own anyway? And if not building your own, how do you see that model evolving? You know, our business is going to get sucked in and then get kind of charged through the nose for them in, you know, three to five years, as has happened occasionally in other services, or do you feel like it's going to be kind of a more open kind of a world than that?

Peyman Mestchian: I know it's very interesting. In fact, you know, I was in a closed session with a number of financial institutions a couple of weeks ago, and this specific question came up and these were like tier one, super, super tier one financial institutions. Already with a lot of their own R& D around this and of course, then you get these LLMs generally available models coming in as well.

Peyman Mestchian: I think the sense I got from the CIOs, CDOs and CROs is that they will definitely adopt. A lot of these external LLMs, the generally available libraries and, and, and, and, and models but they will enhance and customize it with their own. So a little bit like you know, in, you know, if I pick the sort of well established areas, so in the world of credit, you know the established models from Moody's, KMV, you know, and so on that everybody has.

Peyman Mestchian: But then if you go with the bank, they've got their own internal models as well, and they use it for benchmarking and calibration and optimization. There's a mix of internal and external and so on. So I think for the large institutions, they will develop their own capabilities and supplement and augment it with external models.

Peyman Mestchian: As you go down the tiers where they don't have the resources, they don't have the expertise and the labs to develop their own models, the adoption of external models are sort of out of the box. And to be higher in the lower tier. So tier two, tier three institutions. But I think the tier one's will have a combination of internal and external.

Peyman Mestchian: Are you seeing? Which, by the way, gives them an advantage. Be it to be the trading advantage or a pricing advantage. And it becomes an arms race. Who's got the best models. Partly based on the tech itself in terms of how their algorithms are training it. Partly based on the access to data, better data, more data, and they'll have, you know more talent in terms of data scientists and AI experts.

Peyman Mestchian: And therefore, you know, they get, you know those marginal edges that you need to get to, to better price the risk or better price the trade. Is, is, it will be an arms race. Yeah. Just moving away from AI into something that's more core for I push pulls territory. We're doing a lot of work on you know, large language models now, but what one thing that combining combining chat and financial markets workflows is something we've been doing for some time now, and I know you've got a background in risk financial markets.

Peyman Mestchian: We've talked about it before. What do you think are the the obstacles for getting better real time information out to the people that need it in terms of trading and risk from from whether, whether it's third party services or in house risk systems and can chat be a better part of that than it has been particularly for real time because we, we hear it.

Peyman Mestchian: Thank you. A lot that, you know, risk still be reported by email or through batch file downloads. And what do you think the benefits of, of getting stuff in real time? That's the people that need it, but also. What's holding, what's holding things back? Do you think so? Look, you know going back to sort of the decision support mental model of, of, of thinking about this.

Peyman Mestchian: I mean, you're chatting. I mean, if you're chatting, whether you're chatting with a human or you're chatting with a machine, you're chatting with an objective and the objective is to make some sort of decision or take some sort of action. So, so. The chat is a type of decision support, you know, so then it comes a question of what's the source, what's the source there, and of course human based chat is limited to the brain and knowledge of the human that you're chatting with.

Peyman Mestchian: Machine based chat or machine assisted, you know, AI assisted chat is got a vast data lake, almost, you know, almost infinite amount of data to analyze and give you hopefully an enhanced a set of recommendations or for you to make a decision on. So and that's fine. Then it comes to your point around real time real time and then the accuracy and so on.

Peyman Mestchian: So obviously the current LLMs and they will get, they get, they will get better and better in a non linear fashion, but there we do have a problem around some, some hallucinations. From the L N L and msms particularly where there's insufficient data and they make assumptions. And because it's statistical model, the less data you have, the more likely it is that your statistical model is wrong, or you actually give the totally, totally opposite recommendation to what you should and so on.

Peyman Mestchian: So that, that's something that we need to watch out for. And then, you know but I think that will improve. I think that will improve embedding chats as part of a risk based decision making or risk based trading. I think it's here already. It's here already and it will, you know, it's like it will, it will mature and improve.

Peyman Mestchian: The real challenge, the real obstacle to success is to do with Earlier in the value chain, which is around data aggregation. So if you think about the data that needs to be consumed by a model to give an appropriate risk based recommendation, it needs to get data from multiple external sources on, you know, the Dow Joneses, the Refinitivs, the Bloombergs of this world, you know, and news, social media, even now various, you know, external databases and corporate registries and, you know, I don't know, you know.

Peyman Mestchian: Annual reports and so on. So there's a whole body of structured and unstructured data externally. And then you got internal data as well, depending on what the risk, you know, if you're doing you know, you know, if it's market risk, credit risk or operational risk, depending on risk types. So, so the real challenge is look, you know, I want, I want to risk assess.

Peyman Mestchian: You know, Dan today, Dan Eccleston today just extracting Dan Eccleston out of those data mountains of data to extract you as an entity and make sure it's this right that I've picked the right Dan and then, and then risk score you, I need to profile you, profile your behavior, your previous transactions, your trading behavior, whatever I'm doing to profile you that data aggregation and data quality piece.

Peyman Mestchian: Is non trivial and incredibly complex and whatever model you have, if I've picked up, picked the wrong Dan or I've only, you know, I've only identified 50 percent of the features or attributes of you as an entity the output is going to have. You know, a variance on it of certainty back to your point around accuracy and reliability.

Peyman Mestchian: And then how quickly can I do that real time? I mean, to do that real time. So the challenges generally speaking with AI are at the data end of the value earlier in the value chain. And that's how we see it. And a lot of that. Data remains less structured. A lot of that source data, right? It's less structured, it's in multiple silos, in incompatible formats, and you know and it's, you know, good old fashioned.

Peyman Mestchian: You know, reference data, enterprise data management, challenges, and so on. I think we just saw I just saw something this, this morning that that ChatGPT has now made up to date data available. So they're using, you know, 2023 data as a source now. I don't know when they did that, but I think it's going to become available for everyone to start using soon.

Peyman Mestchian: So that's pretty exciting, isn't it? In itself I wonder what the the implications for, you know, additional hardware and processing power and how many more mountains in Norway do they need to always cool with this is an interesting, there's, there's always going to be a trade off, like, is it it's Moore's law, isn't it you know, doubling, doubling capacity every 18 months, and it's been doing it but it's always, Like, like back to where we started, there's always a limitation from that perspective into how much can be done, right?

Peyman Mestchian: We're just going to run into those, those barriers, but it's, it's, it's very positive. There's a lot, there's a lot of positive stuff going on, right? So it's positive. And, you know, basically. If you can invest in data centers, yeah, exactly, exactly is it just, just to wrap up is, is what do you think, you know, that think about, you know, you know, financial markets well and fintechs and that's good, but do you think there's anything that we haven't talked about or, or to elaborate on in, in terms of what we have talked about in terms of like, you know, the next, Three, five years.

Peyman Mestchian: What, what, what the game changing things are going to be? Is, is there anything that you'd hire? Look, I do think, you know, with in, in any industry, in any sort of field. There comes certain moments where you move from evolution. I know that was your initial question on evolution of AI. You move from evolution to revolution.

Peyman Mestchian: I think we are, as far as AI is concerned, we are at the revolutionary point because of those three forces coming together, you know at the same time. So we're at the revolutionary point, which is great, but it comes massive opportunities, be it in financial markets, or in healthcare, or in government, or in entertainment.

Peyman Mestchian: Whatever, whatever industry vertical you think about that the revolutionary opportunities, however with that also comes a lot of risk. So the risk and governance side of it, and I mean, some of it will be policy and sort of, you know, I'm sure, I mean, I know the central banks and the, where the standards bodies are in the financial services sector, you know, from Basel to, you know, you know, very stock exchanges are working hard and coming up with standards and policies around it.

Peyman Mestchian: But I think while that's happening we are leaving ourselves open to poor data, which can create bias. Conscious and unconscious bias, but actually we're also open to misinformation and manipulation. We've seen it, the political sphere in the areas of, you know, voting manipulation and so on. I think the financial sector capital markets are open to that as well in terms of purposeful, bad quality data being, being injected And, and, and therefore then resulting in, you know, manipulated decision making changing the models.

Peyman Mestchian: Yeah. It should not be underestimated. Yeah. Yeah, for sure. For sure. Okay. Well, that's, that's been really interesting. Really fascinating to talk. No, thank you, Swan. Thanks very much indeed. I hope to catch up very soon. All right, mate. Thank you. All the best. Thank you. Take care. Bye.