Introduction

Patrick: [00:00:59] I intentionally avoid the world of quantitative investing on this podcast. The whole point of this format is to learn about many different fields and the vast majority of my time is already spent in the quant world. Occasionally, I've broken this rule because of something unique, including this week's conversation with Richard Craib, the founder and CEO of Numerai. If you listen to the podcast often, you'll have heard me reference Numerai, a hedge fund which blends quant investing, cryptocurrencies, crowdsourcing, and machine learning. Talk about a PR company's dream. One important note, Numerai is both incredibly open and very secretive. You may sense a bit of frustration on my part, but that is only because as a fellow quant who loves details about data and modeling, we couldn't go deeper into the details on the record.

We discussed how Numerai has created an incentive structure to work with data scientists around the world in an attempt to build better investing models. The idea of having data scientist stake cryptocurrency in support of the quality of their models is fascinating to me. Like many hedge funds, Numerai doesn't share its track record, so we don't know if this works. But I hope you, like me, use this conversation as inspiration for how different technologies can intersect.

History of Numerai and How It Works

So Richard, thanks so much for joining me today. You represent such an interesting intersection, which is my favorite kind of investing conversation between traditional quantitative investing, machine learning/artificial intelligence, but also blockchain technology. I've always used Numerai as one of the most interesting examples of people using blockchain for something that seems to be making a lot of sense. So maybe you could begin just by giving us a touch of background on how you came up with this idea, and maybe how that's related to your own history in investing, your own professional background, and then we'll go as deep as possible into some of the interesting features of the Numerai model.

Richard: [00:03:06] Sure. Yeah, the first ideas for Numerai came when I was working as a quant. There was some quant in the fund I was working at, but there wasn't any machine learning that they were doing. So I decided to spend a year researching how we could use machine learning and then I finally found something that I thought was quite good. But at the same time, I was also playing in data science competitions on Kaggle and I thought, "Okay, well, the best way to get really good performance is to get the world to compete on the datasets," and that was true of nearly everything that had ever been done. The Netflix prize was one way better than the people at Netflix could do and then every other tournament on Kaggle. That was one. It was way better than the companies that were hosting the competition could do. At this time, I was also reading about Ethereum and I thought maybe there could be something interesting where finance, which has stayed the same for a long time, could be changed by these two technologies, blockchain and machine learning.

Patrick: [00:04:09] Maybe you could describe the actual process here behind outsourcing, if you will, the analytical or the model building piece. We're going to spend a lot of time on data, and what kind of data it is, and what you do to it to make this possible. But if you could just describe for those unfamiliar the actual sequencing of what you put out there, how you take that, and incorporate it into your system.

Richard: [00:04:27] Well, we have the large datasets that is built up from different kinds of financial data and it's structured in a way where you have these target variables and you can use the data we give to make a model that predicts from the features to the targets. We decided to give away that. No hedge funds really want to give away their data, but Numerai was the first hedge fund to give away all of its data for free, but it contains all the patents of the stock market, but you have no idea what the data is. So you have to use machine learning to basically find a way to model the data, even though you don't know what feature one means or feature two means. And that allows us to share our datasets with anybody without them being able to run off and start their hedge funds with them, but they actually are incentivized to come back and submit predictions to us, which we then trade.

Patrick: [00:05:25] Maybe we could use this as an opportunity to level set on some terminology and describe a generic machine learning process as it relates back to maybe more traditional linear regression or things people might be a bit more familiar with. So you mentioned this idea of features and targets, maybe if we relate that back to what the actual data is that you upload. So when I looked at it this morning, there's a whole bunch of identifiers that could be securities. They're what I would call blind factors. Some of the regularized, which is kind of numbers between zero and one, maybe you could describe what that might represent. I would think about that as independent variables. And then you've got outcome variables or targets or labels, whatever. But maybe if you could describe that generic process of machine learning as it relates to something more simple, like linear regression, that would be a good jumping off point for the rest of the conversation.

Richard: [00:06:09] Yeah, well, machine learning is regression and so it really isn't much more than curve fitting to data sets. So one way to think about it is if you have a bunch of points in a scatterplot on the XY axis, you can find a line that fits those points with a regression, but you actually don't need to know what the X and Y axes are to find that line. And on Numerai, there are 50 dimensions instead of two, I think it's 50 right now, and then the users find curves that fit those high dimensional spaces. Once you can have a regression, you have a model to predict something. And so you're finding classes of points that should be labeled a target of zero, which might mean the stock is going to go down, or classes of points that are labeled one, which means stocks going to go up, and your curve that you find using machine learning to hopefully be very accurate at predicting new data as well.

Patrick: [00:07:12] I always think about these things, especially quantitative processes, as following a similar rubric and what's interesting about Numerai is how differently it's doing certain parts of this. So first of all, I'd be curious if you agree with this general framework, which would look something like engineering the features themselves, doing whatever transforms you need to, in your case, regularizing them, selecting the features, the 50 you mentioned, in the case of machine learning, like hyper parameter tuning, and then you've got to select the models that are going to work that's being built by your audience, ensemble them maybe, and put it into production. Is that how you would describe the stages that are important in this sequence?

Richard: [00:07:46] Yeah, that sounds right. So one thing that people can think about with something like Numerai is we are trying to crowd source stock tips. But really, it's not like that at all. No one's telling us any specific stock to buy. We don't think it makes sense to crowdsource that. It does make sense to do machine learning competitions where you set the problem up in a very specific way. And you ask people to solve that. So the regularization step, which is like normalizing the data, making it seem like something that can be modeled, that doesn't seem too much like non-stationary, then taking that is the part where we let our users. Our users aren't doing any insight into financial data, they just have to do the machine learning piece. And that's the important distinction with Numerai where that's the piece that we want people to work on because that's the piece where the crowd is going to have a large edge over just one person doing it.

Source of Edge

Patrick: [00:08:47] Can you describe that edge in a little bit more detail? I feel like there are two key areas of edge. The first is the data itself and one of the things we find is the more time you spend on getting unique or really clean data set, that drastically improves the outcomes of this exercise. And then the second would be the competition that you're running effectively in an open way, which is ever increasing skill at fitting that curve.

Maybe just talk about the dimensions of that skill. Where's there diminishing returns? If I'm a hedge fund, one of these huge hedge funds, quant hedge funds that has, I don't know, call it a hundred PhDs that are working on custom data set, why wouldn't there just be diminishing returns to the next 50 PhDs looking at the same thing? There are only so many machine learning tactics. It changes but it's not that crazy. So where does that belief that going from a hundred to however many data scientists you have in the world working on this competition? Where is that skill? What's being differentiated?

Richard: [00:09:41] There's definitely only a handful of machine learning algorithms out there that people regularly use. Probably heard of neural nets, and decision trees, random forests, support vector machines. There aren't that many of those. So it is true that anybody could run through each of those and then just ensemble that. But really, you're trying to get as much more creative approaches where there can be certain kinds of emphasis on certain things that you wouldn't be able to do just by choosing a whole battery of machine learning tests. So some of what our users have done, they definitely are using a lot of neural nets, but some of what they've done is very peculiar. We can't tell exactly what they've done, but it ends up having really, really strong performance, so I think it definitely helps.

And then it also helps in a technical sense, it always helps to have more. So if it is diminishing, it still helps to have more. But it wouldn't work if Numerai had to hire every single one of the... we'd be the biggest hedge fund in the world by far, by a 100x if we hired everybody to actually sit in our office and work on this. So we can do things in a better, more efficient way by letting people do whatever they want, use whatever programming language they want, choose whatever algorithm they want. And that makes it very competitive and worthwhile to do it in this way.

The other thing is with financial data, if you have a 52% edge, to turn that into 53 has a really, really big impact on your sharpe. It could be a difference of the sharpe of one or two. So that's why it's particularly helpful in this case. I think you can really have a Numerai for health care or a Numerai for other kinds of data because they'd never have that kind of difference between a 52 and 53 percent edge.

Patrick: [00:11:31] So if there are three parts of this creation of alpha, assuming that's the end goal here, the first being getting the data that you think matters, I want to talk about that, the second piece being what you just described, which is this global network of data scientists working on the best algorithms, and then the third being how you ensemble or create a meta model, I guess, of the best models that people have submitted. Which of those three or how would you assign the importance or the impact of those three categories on your outcomes on the alpha itself? Which is the most important by how much?

Richard: [00:11:32] Okay, wait. Say them again?

Patrick: [00:11:32] So you've got three primary sources of edge, let's call it. The data that you have, what the data is, how you're collecting it, how you're normalizing it. That's proprietary. It's happening at Numerai. No one in the world knows what that data is, the algorithm work that the crowd is doing, and then the proprietary ensembling of those models.

Richard: [00:11:32] I think that data is very important. I think the founders of Two Sigma, I'm not sure if they said it publicly, but they often say that it's the data that's the most important thing, and that is true. If we gave out a problem that didn't make sense in the first place, say it didn't account for the data we gave, didn't account for costs or was tradable only in countries that you actually can't trade in the US by law, anything like that, the algorithms would then find inefficiencies, but that wouldn't really matter because you couldn't actually execute on that. So that's one part of the data. And then the other part is having it be clean and very long.

So I think data is critical and Numerai doesn't allow our users to bring their own data. They have to use our data and so that we are limited by the data we have, but they can use any algorithm. So I think, yeah, data and algorithms are both good. If you have very bad data, it won't work. If you have a very bad algorithm, it won't work. But I think the critical thing actually is not really in the list of three things because the ensembling of models, we don't use anything too... we used to use a very intricate way of ensembling the models together.

But since we released Numeraire, our cryptocurrency, that actually had this massive impact on everything where the people who stake their model with the cryptocurrency, basically, they're taking a bet on their model by betting cryptocurrency. Those users end up having much, much better models. And I think it was actually that piece of game theory, changing the game a little bit, having skin in the game that actually is the is the biggest thing we've done by far because it captures what's going on in normal hedge funds. If you invest in Numerai's hedge funds, I have a whole bunch of my own money in Numerai's hedge funds and the people who invest are investing alongside me, so we all have skin in the game and everything works. But if you don't have that, then things don't work very well and blockchain is actually a neat way to get people to have something to lose.

Using Cryptocurrency as an Incentive Structure

Patrick: [00:14:30] Can you describe that mechanic itself? So maybe say a little bit more about the cryptocurrency that you created, the genesis of that idea, and then this idea of staking. I think some people listening will be fairly familiar with staking, but a lot won't. So maybe just describe that process and why blockchain and the cryptocurrency are uniquely set up to do that.

Richard: [00:14:49] Well, we started off Numerai paying people. It's December 2015 we started paying our users in Bitcoin and it wasn't really anything ideological or something. It was just like way easier to pay people in Bitcoin because our users were anonymous and they were all around the world and so it was much easier to use Bitcoin for payments. But because I had been involved and interested in Ethereum and invested in Ethereum, I was curious about how we could use Ethereum. And a lot of our early investors were people in the blockchain space like Olaf Carlson-Wee, Fred Ehrsam, Joey Krug. These are all my friends in Silicon Valley. So we started talking about could Numerai use its own cryptocurrency in some way? And of course, there isn't really any good reason to have your own cryptocurrency if all you're going to do is make another currency. But if you make an application that needs smart contracts, then having your own token makes a lot more sense.

So with Numeraire, what you could do is stake your predictions that you send to us. If your predictions do well, then you earn more money, and if they do badly, then we destroy your stake. So you'd never want to submit a prediction unless they were good because you wouldn't stake it unless they were good. So it's a big filter for us. The problem with something like Numerai or any crowdsourcing on the internet is that the cost of spam for the spammer is low. It's easy to send millions of emails, and hope that someone clicks on one of them. It's easy to make loads of accounts on Twitter. All of these kinds of things are what we're used to with the internet. But in the real world, because there's a cost to doing things, people do the right thing and that's what we wanted to make happen on Numerai be a little cost. Okay, are you willing to put $10 on your predictions? Because if you're not, we don't really want to use your predictions.

And so that has this really nice impact and why you need a blockchain for this, you might say, "Well, what if you just wire you the money?" But somehow doesn't work because what you're doing with a stake is you're having the blockchain be the escrow because if we just said, "Wire money. We promise to destroy your money," what would that even mean? We promise to give it back. Well, how can you know that promise is going to be honored? If you do it in the smart contract, then you know everything's going to be honored. And therefore, the stakes on Numerai have been really large. I mean, over a million dollars have been staked on Numerai tournaments and a lot of that's been returned and some of that's been destroyed. But it's amazing that you can do something like this now.

Patrick: [00:17:30] In terms of the efficacy, is there a bigger gap between staked and unstaked algorithms or between small stake and large stake? Does the magnitude of the stake matter or is it more of a binary that just when people stake anything, you get much better results?

Richard: [00:17:45] That's a great question. I was very curious about how that would work. Anyone who's not staking, well, we looked at it. It was something like when you get a sharpe of one a half and then if you take the people who are staking, the sharpe goes to 2.1. That's a very big difference. You got to think that the floor sharpe is actually pretty high because the data set we're giving away is already decent, even if you just used regression, so you get like a sharpe of one. But then, to go to a sharpe of 2.1 is amazing and that's just anyone who's staked more than one NMR.

And what you find is, if you say, "Well, what if they stake more than 50 or something," then you get fewer and fewer models. Those models actually don't work that well. So if you're staking a really large amount, it can often be something strange that you're trying. And then it doesn't work very well unless it's balanced out by others. So we actually see that it's good to use any model that's staked with any number of NMR and if you use them all together, then the average is really, really good.

Patrick: [00:18:49] Do you still accept unstaked submissions?

Richard: [00:18:52] Yeah, we do. We let anybody submit to us and there's no cost to making an account or submitting or downloading the data. And the reason you might do that is because you can still get some practice and if you win four weeks in a row with a free model, then you might want to like start staking, buy some NMR and start staking.

Patrick: [00:19:11] Say a bit more about the full incentive structure here for both staked and unstaked. So if I'm somebody that doesn't want to stake anything and I just want to sit in an algorithm and, like you said, I win four weeks in a row, is there a payout associated with that?

Richard: [00:19:24] If you don't stake at all, you can win very small amounts for verifying your phone number and then if you win, you get 1.1 NMR. And the problem with anything when you're giving it out for free is that it's going to be gamed as much as possible. So we have to have those incentives be pretty low to prevent people from gaming it. But once a user starts to stake and starts.. they end up becoming really, really active that some people who've never missed a stake since we started it.

Patrick: [00:19:56] And the payouts there, what's the structure? Do you basically win what you stake? How's the return on staking structured for stakers?

Richard: [00:20:04] Well, as with lot of things with Numerai, it's complicated. Again, you have to be careful about people making multiple accounts. And so any system you design has to be symbol resistant and I think if anybody's been following anything about Ethereum or Vitalik, what he's writing about, he's always talking about can you make something symbol attack resistant. That's because you basically can't rely on any identity.

So on Numerai, when you stake NMR, you're part of a staking auction and in that auction, we're basically trying to allocate money to the people who are staking and you want the auction to be there's no incentive to make another account and submit again to gain that. So we do a variant of a multi-unit Dutch auction, I think. Then people who do their stakes in that way and in that way, the more you stake, the better, but it's also not overpowering to really, really rich users who might stake loads just to try to win the whole prize pool.

Patrick: [00:21:07] How is the prize pool determined, like the size of it? How will that change through time or has it changed through time? And maybe just to put some tangible numbers around this, what would be an example of a data scientist, I don't know if they're all anonymous, but some data scientists that's been active and how much they've taken home as a result of their work on the platform?

Richard: [00:21:25] Well, we definitely actually had some that have made more than a million dollars, which is crazy, and that's because the value of a token has been very high, compared to say how much work it took initially to win some of the NMR before it was launched, some users did incredibly well. Now, we pay about $5,000 per tournament per week, which is about $25,000 per week, $100,000 per month. So it's very profitable to people who get into it. It's quite strange to me that when I started Numerai, I was very inspired by the Netflix prize, which went on for a very long time and it was only a million dollars, and now Numerai has paid out over $8 million since starting.

Patrick: [00:22:09] And like you said, a lot of that, I know if you look at the chart of Numeraire, obviously, it an insane, I don't know what the network value of it was at the peak, but something remarkably high. And as with all crypto, I think it's settled down lower than that. But it's this neat way of engaging a big and smart audience. Again, we've talked about the importance of the data there. I just love to hear your philosophy on what features and what targets to feed this system.

I understand that a whole point here is that you're not going to reveal what the actual data sources are, but just the philosophy around what to choose, where to look for it, how to source interesting data, what targets to tie it to. Obviously, everyone you said before, maybe zero is bad returns and one is good returns or something simple like that. But there are all sorts of outcomes in markets that could be binary as your targets that you're trying to fit to. So I just love to hear your philosophy on how to choose the data, what to feed, and how to choose and feed targets as well.

Richard: [00:23:04] I think for any of this data, you want to make sure that it's normalized in some way. So I'll tell you, for Numerai's data, one thing we, I think, have been open about publicly is that we don't use alternative data or any data that itself already very new. We try to use very clean, very long, structured data. We're already doing a lot of complicated things in one company to then also use very, very complicated, weird data. But once you have that, one of the things you're trying to do is make the past look a little bit like the present.

So if you had a hundred pictures of people's faces, it would be way better if they were all looking in the same direction and had the same lighting. So it's a way better if all the faces were like mug shots because then they're all looking like they're from some kind of similar face distribution. But if there are people looking to the side, people with glasses on, then it starts to be irregular, and it' much better to have regular data. So you want to make the past look something like the present by making the past normal in some way and regular in some way and that's what we're trying to do with the features.

And the same thing for the target variable. It turns out you don't really want to make with hedge funds, you don't really want to make money when the stock goes up, you want to do it on a risk adjusted basis. So you want to make maybe your target of one should be something like a risk adjusted return and that's similar in your goal. With regularizing your training data, you also want to regularize your targets. So Numerai is just very simple binary classification problem on lots of other financial obligations. People actually use regression where you want to model the exact return, but we simplify it by making it 01.

Patrick: [00:25:02] Am I interpreting the piece on data then the correct way when I think, okay, this sounds like if you've got a big, long history of stocks, there's a lot of data on stocks historically, that you're looking at things like financial statements, and price, and things of that nature that are, I would say, let's say common in the quant research world, things like the sources of data that led to the value factor, and the momentum factor, and things of that nature?

Richard: [00:25:28] Yeah, that is the classic quant style data where value, and momentum, and things like that are part of the literature of quant investing. So you have to have those as a starting point, but you can do lots of other things as well. And again, none of the features on Numerai are that. Feature one isn't value or anything like that. But in principle, those the very common features in the stock market and so those ideas are somewhere in Numerai's data.

Patrick: [00:26:01] How much do you subscribe to this old school wisdom of crowd model? I think there are four or five components required for a crowd to be superior to an individual. It's like diversity of opinion. There can't be groupthink, there can't be overlapping. They got to be decentralized. How do you think about that model? Is that something that you thought about when setting this up?

Richard: [00:26:23] Yeah. I mean, people often talk about the wisdom of crowds and they actually forget all of those requirements. You have to have certain things for it to make any sense. So if you pull people on the street, you say, "What stock should I buy," and you ask a hundred people, that's not going to be a good strategy because those people are very correlated with each other and they all just say, "Buy Tesla and sell Snapchat" or something in the city. So you definitely need, mathematically, if you look at what it means to average two things, if they are co-linear, the average is identical to just using one of them. If they are orthogonal and they each have their own signal, then they're actually going to be additive. So these things are very important in the way we think about this.

Patrick: [00:27:15] How many people have staked any Numerai at any point?

Richard: [00:27:19] At the beginning of the year, we used to get 60 stakes per week and now, that's up to 600 stakes per week. So it's pretty cool that it's being used so much and it's way more used than most things on Ethereum. There's some times where there's more stakes in dollars and number than they are CryptoKitties being sold. And so it's really interesting to see that maybe blockchain isn't about consumer things like CryptoKitties, but maybe it's about enabling weirder companies that couldn't exist like Numerai.

Quality of Predictions

Patrick: [00:27:51] It's definitely interesting that on a weekly basis, 600 data scientists would be submitting algorithms to a blind data set. It feels like things like that will happen in the future. Like you said, maybe this isn't well-suited, this structure, to the healthcare world or other verticals, but it definitely shows, just in raw numbers, the power of the crowd. It begs the question for me of as the person at the top of the centralized piece of this, which is taking these algorithms and actually turning it into a strategy for real money, how you think philosophically about the move from old quant, I'll call it, which was form a hypothesis, test it, more linear, sometimes nonlinear, but more linear type, at least where regression type work, versus something now that is way less interpretable.

So if you're working in 50 feature dimensions, we can all visualize that scatterplot, maybe a three dimensional one. But once you get to 50 dimensions, in many cases, we have no idea really what's going on in these models to come up with the predictions. And one interesting question in quant is always, how do you know when it's broken? So if you put a model into production, how long do you give it to work? What are the signals that you look for to unwind it and maybe move on to something else? I'd love to hear your philosophy on whether or not you care about interpretability of what the models are doing and how to know when something's broken.

Richard: [00:29:06] we definitely don't care about interpretability and I think probably have taken that to almost as far as it could ever go. If you think about the way Numerai works, we do not build our own models. We just rely on the people to build the model. So the data scientists who are building on Numerai's data, they're operating on data that they can't see. They don't know what it is. And then when they submit predictions to us, they are not giving us their models. We don't know how they made their models.

Patrick: [00:29:36] Oh, interesting.

Richard: [00:29:37] They are just giving us the predictions. So we're licensing predictions from them, but we never see their code. So what we are putting into production is the raw predictions without even knowing the algorithm that was used to create it. So we don't know what the model is that was built on data that the data scientists don't know either. So it couldn't be less interpretable.

Patrick: [00:30:03] Do you get concerned at all from a supplier power standpoint with that? It surprises me that you don't get the model itself, you're just getting the prediction. So how do you control for the people that build the best model not submitting their predictions anymore in the future? Did you think about having the process be that they actually have to give you their model? I'm curious about how you thought about that.

Richard: [00:30:21] Yeah, it does make a tension, a good tension between us and our users because they really can have this genuine threat of not coming back if we don't pay them or we do something stupid. I like that tension. I think the other companies doing things similar to Numerai, like there's one called Quantopian, where they have all the user models just sitting on their servers and then they just say, "Well, we promise we won't use them without asking your permission." But that's not really part of the crypto zeitgeist, which is like you actually want to be able to genuinely jump out of a system to express a certain thing and be engaged in a trustless relationship. We are not trusting our users with our data. We're totally obfuscating it. Why should they trust us with their models?

I think that's the way things are going and most systems where you don't feel locked in, you end up being more engaged than if you were locked in in certain sense. I remember that, I think with PayPal, there was this thing that the easier they made it to withdraw the money that you had in PayPal, the less people would withdraw it because they always knew there were all these ways of withdrawing. And I think that's a nice way to go with something like this, make it as easy to leave as possible.

Patrick: [00:31:48] You mentioned that you don't build models internally. Do you build a model of models on top of the ones that have been the best performing?

Richard: [00:31:55] Well, we actually did do a lot of complicated things with making a meta model that was based on doing machine learning on the user models to do the ensemble. But it turns out, yeah, the power of the NMR staking, it's very hard to beat taking a simple average of the staked models. We have a few things that help, but that ends up being a really good baseline. And so yeah, it's almost like the users are self-selecting by choosing to stake or not, so they're telling us they should be in the metamodel and what they're telling us is true.

Future of Numerai

Patrick: [00:32:30] So looking forward, what is the most interesting set of things that you're working on that you think might improve the overall process? If the move from not having the cryptocurrency to having a stakable cryptocurrency sounds like one of the major if not the major evolution in the success of the model two a 2.1 sharp, what are the things looking forward that you think will keep you at the edge of the competition?

Richard: [00:32:53] Some big ones that we've done recently, we're improving the staking auction in some of the ways I described earlier where you make it rational only to submit your best model, then that's had a huge impact, taking us even further with our sharp, and then also releasing multiple tournaments. So right now, in Numerai, there's actually five simultaneous tournaments at any time and users can maybe be good at one, but not the other one. And so we're getting more diversity of models because the target variables are different in each of those. So that kind of thing is very interesting.

We released the five tournaments and it only took a couple weeks for the average person to stake in 4.25 of the tournaments. So it's like this amazing thing where our data scientists are totally ready for whatever data we give them and they're willing to try lots of different techniques on all of these tournaments. So I think if you think about another hedge fund, imagine you had a hedge fund that had a very good global equity model, but then they decided we should do a currency thing, that would take many years of hiring people and getting a whole bunch of new things set up before they could even do anything. And for us, it's like, "Oh, we found the currency dataset, let's stick it on and see what people can do with it," and then, very quickly, have the best model in the world for that datasets.

It's really nice to be able to do that. So there's lots of progress we can make in that dimension by just releasing new datasets and every single thing we've done has improved the sharpe, so it's not clear where that will end. And then going forward, I think the one interesting thing that's we hinted at, which we haven't announced yet, is the idea of could we crowdsource the data part? We are crowdsourcing models. Could we get people to give us data? And if you just did that simply like okay, hey, everybody who has a data set, email me, well, you have the same problem of you'll get a whole bunch of bad data, a bunch of spam. Maybe you can use blockchain, maybe you can use NMR to give predictions of a different kind. So that's something that's super interesting and something we've been thinking about for a very long time.

Patrick: [00:35:19] Is that like an Oracle problem effectively in crypto?

Richard: [00:35:23] It is in a certain way, yeah. We would need to trust something. Do you know George Akerlof?

Patrick: [00:35:35] Yes, the name sounds extremely familiar.

Richard: [00:35:38] Market for Lemons. He's a Nobel Prize winning economist. He had this thing about if you have a market where there's lots of bad cars, then none of the good cars will ever be in that market. In financial data, there's a similar thing happening where if someone were to email you and say, "Hey, I have an amazing model that predicts the stock market perfectly," you don't even reply to that email, you don't even bother. Even though what he's promising has insane value, the probability that he's actually got something is so, so low, it's not even worth considering. But maybe if he emailed you and said, "I have this really good model and I've actually staked 10,000 NMR on the fact that it's good," maybe you'd actually want to read that email and spend a little bit of time looking at his data, considering you're playing a different game now. You're playing a game where he's actually got something at stake.

I think that's pretty cool and I think you can extend that to basically where there's market collapse. Markets do not exist for I have signals for stocks. It's always so many questions. Are the signals really going to work? Why isn't the person just trading it themselves? All these things that make it impossible for good markets to form around that, I think those markets would form if you had stakes attached to all of them.

Patrick: [00:37:11] Really interesting idea, so I want to ask a clarifying question. The way you're describing it sounds like, because you mentioned earlier, don't use alternative data sources, maybe stuff that hasn't been around all that long. But this sounds like you might allow someone to come with a predictive signal that's based on their own data, so long as they staked it, or is it just that they have some interesting data and they're staking the idea or they're willing to put a stake behind that if they feed a raw data set into the other part of your engine, the data scientist, that they'll find something compelling? Which of those two is it?

Richard: [00:37:46] It's really more just if right now on the website, when you upload predictions, you have all these abstract security IDs, you don't know what they are. But if you could upload to Numerai's website that you think Apple is going to go up, and you think Google is going to go up, and you can actually give real ticker symbols, then you could be generating that up and down signal from arbitrary data that you have of data, whatever.

Patrick: [00:38:13] So it's both. At the end of the day, the submission is still a signal, not just a raw data set. So someone couldn't say I've created this amazing, whatever it is, data set on all companies, but I don't know what the hell to do with it, so have your data scientist figure it out. Instead, it's no, just do whatever you want to do, but you're submitting a signal and you're staking something on the predictions embedded in that signal.

Richard: [00:38:33] Yeah.

Patrick: [00:38:34] Yep, understood.

Richard: [00:38:35] I think like opening Numerai up to let other companies use it or other industries, I think it's just so the wrong approach. Instead, it's more about individuals. It's like if you're an individual and you have some data that you want to follow the protocol to upload it, then this could be a really good place to put it, especially if you're willing to stake on it.

Patrick: [00:39:00] I would love to hear your impression or take on the rest of maybe even specifically quantitative investment community. You said you came from sounds like a more traditional hedge fund where you were working on machine learning when maybe at the time, not many other people were. You mentioned Quantopian, Two Sigma, you've mentioned some of the names in the quant world. I'd just be curious to get your take on where you think that world is in relation to some of the topics that we've discussed. So how many of the big players are actively using machine learning techniques versus that old traditional way of thinking about quant?

Richard: [00:39:35] Yeah, I think, definitely, machine learning is not as well-used as people might think by the big funds. They might have teams that are doing machine learning, but probably none of their stuff's actually in production and I think places like Renaissance don't use machine learning from what I know from some of the people who work there. That's just not what they do. One of the reasons for that is that they just have an edge. If they have a really good data edge, and suppose they've been just collecting stock market data long before anyone else and actually storing it in databases long before everyone else, their data edge is so big that they don't need the modeling edge to be very big. That's counterintuitive, like the brand of Renaissance. It's not like we have the best data. That's not what they're saying. They're just like, "We're the smartest people," but maybe they just have the best data.

Patrick: [00:40:26] It's why I asked the question earlier about if you had a pie chart of the importance of the data you have versus the... I guess it's the informational versus analytical or modeling edge that everyone always talks about, which is more important. It's interesting that maybe places with all these PhDs, maybe it's a data advantage, not a modeling one. Really interesting. What else is exciting to you in the hedge fund world? It's interesting to hear all of these very crypto-like philosophies applied to a very old school unchanging model. So are you encouraged by that overlap? Are you frustrated by it in any interesting ways? I'm assuming you're set up with prime brokers just like everybody else and you have the same legal teams as every other headliner hedge fund. I'm just curious of your take on the intersection of those two worlds and any surprising reactions you might have had to them.

Richard: [00:41:07] I think one of the interesting things about the crypto... when you're designing something in crypto, you're really thinking about what's the best thing, you're not thinking about making money. This couldn't be more true of, say, someone like Vitalik. When you're thinking about designing proof of stake or something like that, the last thing on your mind is money. It's just like, how could you get the system to work and then all these different actors to behave in a certain helpful way. I don't think anybody has thought about the hedge fund industry with that lens, so I'm trying to be the Vitalik of hedge funds and basically look at the stock market and say if you're an alien coming to earth and you were like, well, clearly, capital allocation is super important problem. You need all the money to go to the right places, otherwise, society won't move forward and technologies won't be created and so this is like a really important thing.

And the way that the humans have set this up is well, let's have a huge, weird zero-sum game where there's just all these smart people from the top universities joining hedge funds that are identical to other hedge funds in every single way, except that they're friends at one and they're at this one, and they're all buying the same data, and none of them are sharing any knowledge about what's going on. It seems like a schelling point. It doesn't seem like a super Nash equilibrium. There's probably some technologies we could use to make all this much, much better.

Patrick: [00:42:49] Can you describe a schelling point? Just not to take it for granted.

Richard: [00:42:49] Something that is an equilibrium. I don't have the perfect definition in front of me, but something like it's an equilibrium but is like a local...

Patrick: [00:42:50] Local optimum or something.

Richard: [00:42:50] It could be way better if certain things changed. That's what Numerai has been doing where you have things like, "Well, we want you to help work on this data with us, but we don't want you to run off with the data." And if we could just solve that, we'd have a completely different system. We can actually have a distributed hedge fund, which has never been done. So that's the problem we first worked on. And then we're like, "Well, now how do we get people to really express their belief?" And then we're like, "Well, what if we use Numeraire and made a cryptocurrency that did that?" So the game design of the company is the company. We're not trying to do anything except make a system that is resilient and long-term going to be way, way better. So yeah, that's my take on the hedge fund industry. I like that in a way no one else is coming at it from that perspective.

Patrick: [00:44:03] One final question on your strategy is... and I'm obsessed with marketplace businesses in general. I just think it's so interesting when a business has to build two sides of a market effectively, yours being data scientists on one side, investors on the other side, and you're the thing in between, that the concern would be somehow, a competitor comes up and spins up something that pays out more or, in some ways, caters better to the data scientists as a better audience building strategy. So I just love to hear your thoughts on that, like you reach data scientists, how they become aware of Numerai that's very intentional versus just word of mouth. Maybe it is just word of mouth, but if it's not word of mouth, like strategies you have for building that audience and for keeping them happy, given that they represent such, I think, a large portion of your advantage.

Richard: [00:44:49] We didn't do any advertising or anything like that, but Numerai, because it had all these unique properties ended up-

Patrick: [00:44:56] It's like a PR dream.

Richard: [00:44:58] Yeah, it became like us... there was a lot of press about Numerai. It's very interesting to me how if you ask a Silicon Valley software engineer, name three hedge funds, that that Numerai would maybe be one of them. It's crazy. But yeah, that was a big part of getting users. Nearly everything we've been doing has been zero to one, so we have made some mistakes, and some of our users get grumpy with us about certain things, and we have been innovating faster than we've been maintaining, in some sense in some parts of what we're doing.

But I do think that our users are, at any given time, if you're holding NMR, it's because you think it's better than selling it. We have loads of users holding NMR and staking it every week and it's up 10x since the beginning of this year. So I like where we are in terms of all the important things. I've never understood cryptocurrency valuations, so it's hard to think clearly about that when they're so correlated, even though Numerai is so different. But yeah, I think it'll be hard for another company to come and do something similar to Numerai. I think there's a lot of things under the hood as well that can't be seen.

Patrick: [00:46:16] Well, definitely an interesting intersection of some of the more important technologies of our day, so it's been interesting to hear about everything you're doing. My closing question for all my guests is to ask what the kindest thing that anyone's ever done for you is.

Richard: [00:46:30] Coming to mind is the early parts of Numerai where I came to Silicon Valley without any connections here. I was coming from South Africa. Howard Morgan, the co-founder of Renaissance Technologies, was our first investor and he has a similar thing, I think, where he's not in a time in his career where he wants to just invest in things that are nothing but making money. He wants to invest in very interesting things and things that could be really interesting for the world and society and things like that. So he was extremely kind to me by investing in Numerai.

Patrick: [00:47:10] Wonderful. Well, thanks again, Richard, for your time and hope to catch up as you guys continue to evolve.

Richard: [00:47:15] Yeah, thank you.

Created by @SamMeuret