Introduction

Patrick: [00:00:49] My guest this week is Eric Sorensen, the CEO of PanAgora Asset Management, which manages more than $46 billion for clients across a variety of strategies. Eric began his career serving in the Air Force as both a pilot and instructor in high-performance jet aircraft. He then accumulated 40 years of quantitative research and investment experience with a PhD along the way. Please enjoy our conversation on the changing landscape of quantitative investment strategies.

From Fighter Jets to Finance

I thought we'd start all the way back. You have a unique background in that you spent some time in the Air Force. And one of my all-time favorite books is a book about John Boyd and sort of the OODA Loop and all the interesting things that he learned. And I know you were an instructor in the Air Force for awhile. And I would love to use that just a brief story of your time there and sort of the foundational lessons that created the springboard for the rest of your career from the Air Force.

Eric: [00:01:37] I appreciate that. I was in the Air Force in part because of the environment. There was something called the Vietnam War. And I was an undergrad in college at the University of Oregon, and I joined ROTC. And my father had been a military officer in the army in the second World War, and we had a kind of a proclivity for serving. So I went to pilot training and that's a one-year program and then soon after became an instructor pilot in a high-performance jet airplane.

Patrick: [00:02:02] So tell me about what kind of planes you flew and maybe the early days of the instructor - I'm especially interested in that stage.

Eric: [00:02:09] Well, thank you. I flew an airplane that we used in advanced training for people that are getting their pilots degree in the United States Air Force, their wings, and it was the third plane that they would have been in. And so the last one before they get commissioned as a pilot, it's called an F-5 or T-38. A fantastic airplane. Northrop built it. Still in use. The one we flew carried 4,000 pounds of fuel. And when it didn't have fuel, it weighed 8,000 pounds, it's just a rocket - no lift - just a rocket, it's called supersonic and high-performance.

And the thing about an aircraft that is somewhat similar to the business you and I are in today, it's a machine with limits, rules, great performance, but you can overextend it. But it's also operated by a human. So there's nothing like being in a high-performance aircraft and pulling 5 G's with your right hand on the control stick, and your left hand on the throttles and being forced down in that seat and just that feeling. And I don't think the machine could do that by itself.

So I've often thought about what we do now; it's a combination of intelligence, human intelligence, decision making, thought process, instinct, emotion, and using whatever the latest tools are to do that. And I really enjoy that.

Patrick: [00:03:18] It's an amazing analogy. And maybe it's one we'll kind of return to through the conversation. I'm curious for new pilots that were operating this sort of high-performance machinery for the first time, how you taught them to transition? And there's an obvious analogy here, which is fundamental analysts they haven't used quantitative systems and maybe they should. So what are the key steps in getting someone used to working with such a high-performance machine?

Eric: [00:03:38] That's a very good question. First of all, these students who - I call them students in quotes - they were like one year younger than me. I had been one a year earlier and then I was an instructor for about five years. They would come into our flight where I was an instructor pilot, and they had just finished training in another jet, which was subsonic and had very different aerodynamic characteristics. So this is a new experience. This thing is going 250 or 300 knots by the time you get the gear up - I mean, it's fast, it got afterburners and so on.

So the first couple of flights, they're a little nervous. But for an instructor and for their program, they have to go solo relatively soon in that aircraft, like after maybe six or seven sorties or missions. And as an instructor, you have to make a judgment. Is this person, in those days, there were just men, now there are women, can he do this? And so in a sense, I learned early on in my career, how to discern that and then to take the risk to empower people. Because you get out of the plane, out of the backseat as the instructor, and you watch your student go and you keep your fingers crossed that he doesn't crash and burn.

Patrick: [00:04:46] Were there common traits of whether they're temperamental traits or otherwise that made for very good pilots relative to say more mediocre ones?

Eric: [00:04:54] Mediocre and very good. Let me talk to that. There was a huge dispersion, even after people graduate in what you consider skill and ability. And upon graduation, some are recommended to go to be aircraft commanders or instructors, and others had to serve some time as a co-pilot. Even though they all were above a certain minimum, that's part of it.

The other part of it is; I don't think there's anything you can point to. It's not athleticism. It's not necessarily intelligence. It might be being able to do well under pressure. There's a series of check flights you have during a program like that. And thereafter you have them even as a fully rated pilot. You get inspected by the inspector general. You're put in there, no notice, and someone's grading you. And, I mean, you're going 400 knots and you're doing different maneuvers, you got to be able to do that. But I don't think it relates to any other area - physiological or otherwise. We had big people, small people, large, we had good athletes, bad athletes, we had smart people and we had kind of average intelligence. You never know.

Patrick: [00:05:49] Fascinating stuff.

Eric: [00:05:50] Yeah. Really is interesting.

The Evolution of Quantitative Research

Patrick: [00:05:51] One of the things I've been most excited to talk to you about is that you have basically seen the major stages of quantitative equity research evolve. It was a niche thing for a long time. It's now probably a dominant trend in markets, in equity markets, in all markets. So I'd love for you to describe kind of in your view what the major stages of that evolution have been - sort of dry history. And then obviously we'll spend most of our conversation talking about today and the future.

Eric: [00:06:16] Okay. When I got to the University of Oregon, it was just the beginning of an explosion in financial economics, financial research. Probably the prior 20 years, most of the really good, they were economists, not necessarily "finance professors" and there were two areas in particular that really began to shape the literature and the way people conducted themselves with respect to doing dissertations and so on.

One was the concept that you ought to earn a rate of return in markets that's consistent with the risk you take and formerly it's the capital asset pricing model. And I like to think of that as asset pricing equilibrium. Jack Treynor, who did not get a Nobel Prize, who I knew well, he's now no longer with us was a pioneer of that, and Bill Sharpe and Jan Mossin and John Lintner got Nobel Prizes for that. And basically that was an area that was new but it was descriptive. It was like, "here's the way the world ought to look."

And then there was another area of literature coming out of Chicago and mainly Gene Fama and others, which was also descriptive. Not asset pricing equilibrium but asset pricing competition. It's hard to win. Because if information is readily available, entry costs are relatively low and you can make money, maybe you want to try this.

And I remember one author way back and I've used it a lot said, "If there's a lot of competition and there are some costs to entry, the game maybe worth winning in terms of if you get the right result, but the game may not be worth playing because of the expected value, the outcome is negative." And that's really interesting. So then there was a body of literature on proving or disproving efficient markets, those two things.

Patrick: [00:07:55] So talk about your insertion in this stream, the research that you were doing, obviously we're both active managers. So we disagree with some of the things that these Nobel laureates have put out there. So talk about how your academic research started to dovetail into practitioner world.

Eric: [00:08:08] And there were two other areas of research. The fourth one I'll mention is where I spent most of my time - call them areas of innovation. One was the whole area of, well, empirical studies - so maybe you can do it - anomalies. So the academics had to explain it away as this couldn't happen, really small-cap or earnings surprise or whatever and behavioral explanations, because people don't think right. They sell at the wrong time, and buy at the wrong time.

And then there was my area, I did a lot of research on pricing. So I call it basically econometrics of asset pricing. Just straightforward looking at, "Can we get better at actually knowing where a bond ought to trade or a stock ought to trade?" So I did a fair amount of research, quite a bit of published academic research, for a variety of purposes where I had to use pricing models. And these were based upon multiple regression and the variable of interest would be the price of an asset. And then the characteristics that would determine whether that price is high or low. And that was an area, and that really led to a lot of the empirical work that now has found its way into investing, I think.

Patrick: [00:09:10] Can you talk about that early feature sets - the outcome variable is price - I'm always fascinated by how people decide what features to include, what they do to normalize and scrub those features. So in the early days of the research, what sorts of things were you looking at?

Eric: [00:09:22] Well, my dissertation was on the question of municipalities that issue bonds, the municipal bond market is a fairly large market and there are investment bankers involved almost in every case and distributing those bonds. And there are two ways it happens. One is, there's a lead underwriter who negotiates with the issuer, whether it be a government or a county. And in other cases, the jurisdiction requires competitive bidding. Like you would on any other contract to build a building.

So there was a negotiated way of selling these bonds and a competitive bid. So the question would be; which one is better for the issuer to minimize their borrowing costs? So I had to develop a model and others had done some of this too. And I was in a group that we did this, where the dependent variable was the yield or the price of the bond or something like that. And the independent things were factors, data factors, multiple regression - obviously the credit rating, obviously some other information about the municipality, the slope of the yield curve, the structure of the issue. And they're all basically tranches with different maturities priced together.

So you build these models. Number one, they had to be things that made sense. And today, even in the business we're in, at the firm I'm in, PanAgora, we have to have intuitive causality when we investigate things. Doesn't mean we don't use fancy tools, but it isn't just price versus whatever we think can correlate with it.

Patrick: [00:10:43] Can you talk a little bit more about that trade-off? So that's a really interesting question in today's market, which is certainly some firms would hold out the fact that they don't need to be able to interpret their models as a relative competitive advantage. So talk about the trade-offs between that kind of spectrum of interpretability.

Eric: [00:10:57] There's probably a trade-off, perhaps both work. I would think the one that doesn't have an understanding inside it, or - to use the machine learning terminology - isn't supervised, it's unsupervised, just finding correlation structures embedded, and then repeating, repeating, repeating with algorithms. And there the objective is to minimize the air. What if it isn't working and it is trained on historical data? You're really in a quandary as to why. Secondly, what do you grab? What data do you go for? How do you decide actually what to look at?

So our equity team and our multi-asset teams are all, if you will, co-existing. The researchers are also portfolio management and vice versa. Some are older, some are younger, some have different talents. They watch these markets, they watch the performance results of our strategies, and they're all the time looking for new inputs, but they have to kind of make sense, otherwise you could just go after anything. And there's a budget, if you're buying "big data" - I call it smart data, we built a lot of our own, it's expensive. So you have to really make a, if you will, a business judgment. I don't know how you would manage that without a belief. So I go back to the question, it's just a different belief.

Patrick: [00:12:06] So now talk about the early days of your practitioner career. So you start taking sort of this skill set and applying it to actual markets and actual trading.

Eric: [00:12:14] I was doing these research projects that were pricing either, not just municipal bonds, I went into corporate bonds and yield curve modeling, as well as things like preferred stock and convertibles with different features. And I got involved with some investment banking people that said, "Well, what does it cost us if we put a sinking fund feature on a bond?" Well, you had to have a model to say, what would the bond sell for, with or without? So to me, this was like in my sleep.

Patrick: [00:12:37] Catnip, yeah.

Eric: [00:12:39] So I found out you could marry - and I love to be creative, I'll come back to what I think the motives are of why we're in this business. I think there are two, why we run our firm, why you run your firm. And the motive there was to get to the intersection where you get to research and discover things and someone likes it and it's useful, it's practical.

So after time passed, I became department chairman at the University of Arizona Finance, published quite a lot. I took a one-year leave of absence and went and worked with a small firm. And then after that, I never went back. I found Solomon Brothers or they found me in 1986 and they needed to build a quantitative equity group. They had none to speak of. Like what they'd already done in a major way with their bond analytic group, not the proprietary stuff, but Marty Leibowitz's stuff on bond portfolio analytics. So I went there and started that business.

Patrick: [00:13:23] So, many listeners will be familiar with sort of the simple evolution of what we call risk premia or what I might call like the core primary factors, things you've already mentioned, some value, momentum, things like this. So obviously that tends to be one thing that many quants use, certainly not the only thing. And it's become increasingly common. In the materials you sent me, you've got this great thing called the 5 C's. And so I'd love to hear what the 5 C's are and use that as a means to talk about kind of today's research.

Eric: [00:13:48] Now, that's a great question. Just as I observed the life cycle of strategies in competitive markets with a lot of motivation for people to participate when something's working and then seeing it ultimately not work for a period of time or even just ceased to exist. It strikes me that first of all, you have this set of people or minds that are creative and they like to discover, and they think this might really work and if it is truly creative and it works, that's great.

And you're either adding value or you're controlling risk better and so on. But it's a very competitive space and people begin to copy it. We wrote a book at PanAgora about 10 years ago, myself and two colleagues on sort of a follow-on on to Grinold and Qian, how do you do quantitative management? We have a lot of stuff in there, which we'd formally published in some pieces in some journals, which really disclose what we're doing - what we considered to be sort of state-of-the-art at that time. Well, you kind of do that and people find out and copy it.

So you have a situation where there's creative idea, it gets copied, money flows in, it gets a little congested or crowded and ultimately there's convergence on a limited supply of potential assets you can buy or sell. And then the Federal Reserve, or there's some external catalyst says, "Wait, things, aren't the way we thought they were." And people begin to sell. And then there's ultimately capitulation because it becomes illiquid.

The first one I saw, I observed up close and personal with Solomon Brothers in 1987. In the day the market was down 22%, forget which day, it was in October - it's always October. And it was associated with a very good strategy, started by three - two academics and one business person from the West Coast; Hayne Leland, Mark Rubinstein, who I know both of them and another guy, John O'Brien, LOR, and they had this thing called portfolio insurance. Think about our business, you got to have a great name.

Patrick: [00:15:31] It's not death insurance it's life insurance, right?

Eric: [00:15:33] There was a label, when I was at Putnam for a brief period, where they had on a product, which was like "sounded too good to be true." But anyway, that strategy was to use the futures market, in their case to advise pension plans to hedge their equities, and there was other people doing it, actually using options, which was very interesting. But anyway, that strategy became fairly large. And one day the market was down and these portfolio insurance programs kicked in, futures gap down and with index arbitrage, the cash markets gap down and so on. And by noon, the market's down 10%. By two in the afternoon, the New York Stock Exchange and others are calling our head trader in equities and Goldman's and Morgan Stanley's, saying, "Don't back away from the market" and it was down 22%. So there was ultimate capitulation.

One of the interesting things there, you could have hedged in downdrafts using options. And one firm in New York did, I believe the name of the firm was BEA and I met the principal at some point in time. I thought, very bright guy. And he was doing this for pension plans, or his firm was by selling out-of-the-money puts in the S&P, which were reasonably inexpensive. I don't know what the structure of implied volatility pricing was. Maybe it was flat. Maybe they were even cheaper than an at-the-money put. Probably. Ever since that day, there's been a skew in that pricing structure. The out-of-the money puts have never been cheap again because people realized that was probably the best way to preserve wealth because it's contractual. It was an insurance premium. Anyway, I saw that. And then other episodes of emerging market debt crises, and Long Term Capital Management, and even to some extent, the 2007/8/9 equity thing, there was a lot of money and you mentioned earlier it, crowded. So I just think that those things will probably, unfortunately, repeat.

Patrick: [00:17:13] So I want to take these 5 C's and apply them to something like value investing, for example. So you've got the kind of Andrei Shleifer limits to arbitrage idea that for some of these strategies, maybe they go through this 5 C's cycles many times, but their long-run expected excess return is still positive. And so I'm curious to get your take on what differentiates, let's say a signal or a factor that will have, despite knowledge of it, like value will have persistent outperformance versus something like Long Term Capital or some other strategy, which is fully arbitraged away through the 5 C cycle. So what's the difference between those two?

Eric: [00:17:46] Well, the value proposition, I think is real one in a long-term sense. I do believe that things become mispriced. And if it's a viable investment vehicle, like say the stock market or the bond market or subsets of that, there is mean reversion. So if I were, 30 - 40 years ago, starting a money management firm, I'd be an equity value investor, and I'd make my clients allow me to keep their money for five years. And I would always have higher wealth. There's a thing called stochastic dominance. Can you accumulate more wealth for your client over a five-year period.

if I were, 30 - 40 years ago, starting a money management firm, I'd be an equity value investor, and I'd make my clients allow me to keep their money for five years.

The trouble with signals sometimes that are short-lived, I mean, there are periods of time where they're not working. We've been in that in the last couple of years in equities, in quant equity, and maybe it's crowding, it may be other considerations, other influences. You mentioned things that ultimately fail and don't work, and that does happen too. But I think you have to have a philosophy that this strategy will work over time and you can tweak it.

Patrick: [00:18:38] And the way to think about this question is, how to know when something is broken. So, one, I'm curious whether or not at PanAgora you combine signals, maybe like value that you think in some way will work forever with those that maybe you expect won't work forever. So you mentioned smart data. There's plenty of examples of information arbitrage that go away. So how do you think about identifying when a strategy or signal is broken and should be removed from the process?

Eric: [00:19:01] The categories of these elements - the drivers I think - are good, they're broad. In equities, they are: sentiment, valuation, quality. You can call it ESG. The metrics you use have radically changed. I'll come back to that a little bit. The other part is portfolio construction. So there's lots of strategies - macro risk premium, risk parity - strategy really is heavily dependent on how you construct your portfolio, how you risk weight. That's a pretty good proposition.

My colleague Eddy Qian will tell you every major asset class has the same sharp ratio over any 20 year period. But during some periods, some of them really have great sharp ratios and others are down. But if you know that and you risk weight commodities, equities, global markets, yield curves, bond markets in various countries, it really has a nice distribution. It doesn't have the fat tail on the left side. So that's a construction. I think that's fairly fundamental, but you have to stick with it. You can't just sort of discard it, if in fact another asset allocation scheme like heavily oriented toward equities in a bull market looks "better."

And the other side, the equity side, you have to use better data. So I did the first, I believe, the first quantitative equity ranking system that was put in production on behalf of pension clients for the S&P 500. We did it from Solomon Brothers and it was the same stuff I'd been doing for the prior 10 years as an academic except applied to equities. And every stock had a valuation. And that valuation was based upon projected growth and normalized earnings, but a factor model, which would adjust the way you actually, if you will, give a uptick or downtick to the multiple - but I put it in a discounting mechanism, like a long-term DDM. But the factor model was adjustments to the discount rate based on risk and volatility and stuff like that.

And the reason it worked really well, for every stock, it was slow-moving. So a stock that had a great value wouldn't change overnight and the quintile spreads were reasonable. You'd buy the top one or two deciles. Very simple portfolio construction. And it worked because, well, not many people were doing it that way. And it was simple. The way it worked was, this was called the e-model - a Solomon Brothers model - and the way it worked was, we had internal index funds and they would carve out some of their money and the internal counterpart to me, the person running the index fund would implement what we told them. And so they tilt the portfolio toward this model. And then it turned out it did better than their S&P 500 index, more or less over time.

Patrick: [00:21:23] So the early days looked that way.

Eric: [00:21:26] Yeah.

Patrick: [00:21:26] Obviously today, I'm sure looks a lot different.

Eric: [00:21:28] It does.

Quant Today and Going Forwards

Patrick: [00:21:28] So there's two components to this that I want to dive in some detail on, the first is the data, and the second is the models used to find patterns in the data. So simplistically, you have linear regression on top of book value and in more complex terms, you've got deep neural nets and random forest on top of all sorts of interesting smart data. So talk about maybe kind of focusing on today and looking forward what that looks like now. So I think the concepts are similar. Be systematic, be good at portfolio construction, et cetera. Like you said, you've already said it several times, you need to evolve. So talk about what that means for your firm today.

Eric: [00:21:57] I agree, Patrick, they are two aspects to try to win today. One is the data you're accessing, how you implement it, where you get it, how creative you can be with it. And there's a lot of data today. That's one we'll come back to that. And the other is what are the tools you use to actually then extract value from that data? And they are the advanced, if you will, statistical techniques and other types of things that are needed today, non-linear, hierarchical, and maybe machine learning in some sense, in a real sense. But the data, I give this talk about, we call this smart beta and big data. So I call it the smart data and big beta, beta is too big and the data have to be smart. It's a $50 billion smart. I said to a group of reporters a few years ago, I said, "The vendors of what's now called big data. The average of their revenues is enormous. A lot of those providers really don't have well-defined business models. They just happened to be niche-y. They know some data." So the same things can happen there. If you're buying from a vendor and other people have access to it. So we are aware of that.

The other part of it is you can buy some or you can actually then create by different sources. So the creativity there comes with an investment team that really knows where the data sources are. First, you need to know what drive stocks. One of our groups have, I think a pretty good model for biotech. It has nothing to do with financials. We've had this for 10 years. It's a part of one of our strategies. It's now a standalone biotech pharma portfolio. Biotech has to do with drug trials. That's a coin toss in some sense. Well, it's not really because we'd call in the Goldman Sachs and Morgan Stanley analyst, the biotech analysts years ago and say, "What do you look at?" They said, "I don't look at prices. There's no earnings. We look at the FDA approval process, we look at what they're working on and we visit them."

The term I use is expert systems. This team build an expert system to mimic what that analyst knows about those handful of stocks, except we're not as deep, but we know it more broadly. So that data set we've perfected over time. And there are other examples too. And even having said that you have to continue to be ahead of the game. So there, and I don't know actually how we're running that today, but early on, I know it was compounds. And what cohort group are operating the same kind of discovery process of drug trials. What's the likely success? What kind of a Markov chain analysis, probabilistic model, like: these biotech firms really low odds, these above-average odds. And so you buy the above-average ones.

So that's a data set that was unique. I call it smart data and I say this about it. (a) It has to be intuitive, it has to make lots of sense because analysts from the sell-side are doing it, and (b) you have to be able to gather it and collect it. And if you can do it proprietarily, that's really a big deal. You're not buying it from a well-known vendor. And (c) we believe most of our work, and maybe others are different, we have to have a reasonable horizon for the payoff. So price tomorrow is very rarely a dependent variable on things we do.

Patrick: [00:24:53] Yes. Non-stationary.

Eric: [00:24:54] Yeah. It's just all over the place.

Patrick: [00:24:55] So that's a great example to talk about kind of mixing expert systems let's say. So if you've got a biotech system, which is hugely different from somebody relying on financial statements or something in another sector, and so you've got, let's call them competing models or signals in terms of relative attractiveness of the asset, the underlying asset, how do you blend those two things thoughtfully in portfolio construction? So if you're in a portfolio, I certainly understand if it's a standalone biotech fund you're comparing versus other biotech companies. But if you're comparing a biotech stock to a consumer staple stock, and those two stocks are vying for weight in the same portfolio, how do you think about that integration?

Eric: [00:25:29] We, a fairly long time ago, we were using a technique for looking at the signals - how they co-vary and effectively we were trying to optimize the Information Ratio. And through time, as you look at the inputs, you can actually get a weighting scheme that comes right out of that. So that weighting scheme will give you the optimal IR or whatever factors you're putting in, given the history of those factors. So as they change that weighting scheme will change over time. So one of them is just the alpha, if you will, the weight you put on a signal within the composite ranking.

Patrick: [00:25:57] So if you're optimizing for IR something, we always struggle with a little bit, which is often the best way to do that is to reduce tracking error not to increase the numerator. So how do you guard against that?

Eric: [00:26:07] Well the other part of it is, is the risk, what you call risk. And so when I was doing these early models for enhancing the S&P, and then the real players came in, Wells Fargo, which became BGI State Street. They came and saw me. They saw my counterpart at Goldman on the sell-side, how do you do this? And they made businesses out of it. For us it was just a niche-y thing. And it was fun. And we made some money. We got paid by the way, by transactions. So we weren't fiduciary. And it was a lot of fun. They perfected a lot of this and the rest is kind of history there, but it evolved.

Patrick: [00:26:35] So that's a good example of what you call smart data. If you're willing to share, I'd love to hear other examples just to continue to contextualize this for people thinking about this idea. But then also how you think about applying the very buzzword driven new statistical techniques, which are just that - they're beefed up evolutions of very old techniques. We were talking about my friend Jeremiah Lowin before we started recording. And he always urges me to think, if you hear something really fancy, just think stacked regressions, right?

Eric: [00:26:58] I heard that, yeah.

Patrick: [00:26:59] Sort of regressions all the way down. But how do you think about using those liberally? Talk about a powerful machine, like what we started the conversation with, responsibly, and then marrying that with these smart data sets.

Eric: [00:27:09] The classic quantitative or fundamental portfolio, the manager or team or process ranks the security. So you're going to buy the higher-ranked securities. The question is, can you use advance sort of techniques to identify value and seek relationships and get better inputs, right? And the buzzword today is machine learning or artificial intelligence - it is a term that I don't like to use with financial markets, but it's used. We have so many AI experts today in our field, I'm going to digress for a moment here, the way I think about that. And I think some of my colleagues would share this.

First of all, you go back to the definition, the definition by Alan Turing in 1950, if in fact, the machine, in those days the computer, which is mechanical and somewhat artificial, maybe in some sense, if it can produce results that are indistinguishable from a human, then it's artificial intelligence, maybe even better than the human. Of course, that's been shown to be the case with a chess-playing computer versus a human; Go, driverless cars.

You think about all those applications. The two things I think come to mind, one is regardless of how many possibilities are in a chess game or in a car driving down the road, they're finite, and two, they're rules-based and deterministic. They're rules. So think about an expert system that can actually do that. Maybe even facial recognition. You get into financial markets (a) both of those things break down. You have Brownian motion, you have stochastic processes, so almost an infinite number of outcomes of a security price. And (b) is it rules-based? Not day-to-day. So that's an issue.

But machine learning is a tool that actually can get inside and maybe come closer to getting a solution than if you were doing it just with your own head and some simple data. And I think some of the applications that are really interesting there, you mentioned stacked regressions. We started using decision trees back when I was at Solomon - my colleague then Joe Mezrich, who is still a sell-side quant, I think the best sell-side quant on the planet. I mean, we worked together since '87. There are others I worked with too that are...

Patrick: [00:29:14] Probably the only person that I read with regularity.

Eric: [00:29:16] He's amazing. He's a good friend of mine. Anyway, we started using classification trees because they're nonlinear and hierarchical and they're rules-based. And the test statistic on R-squared, it's like the probability you've got the right answer and we use regression trees, but there was too much noise in that. So we classified with quintiles or deciles. Well, today the work he's doing and others, they use multiple trees, random forests, or they'll do multiple trees through time so that structure can actually learn and can adapt to changes in what causes return.

Patrick: [00:29:45] With boosting, yeah.

Eric: [00:29:46] Yeah. So I think that's a pretty good application. As long as the engineer or the designer knows the potential limits and pitfalls. The trees have to be pruned back. You can't just let them fully populate.

Patrick: [00:29:57] Back to your 5 G analogy.

Eric: [00:29:58] Well, yeah. Because they over-fit. And then, you've got spurious results and you can't count on it necessarily. The other's natural language processing, which I'll come back to.

Patrick: [00:30:07] So let's talk about that a little bit. So why and how is that useful today?

Eric: [00:30:11] We have some colleagues that have actually done quite a bit in that, and we have some applications and it's actually in our models. One application is to use the reading of filings, corporate filings. And there was a study done by a couple of professors at Rice - they were mentioned, Renaissance, AQR, but I think we were number two. They got information from the SEC as to who's hitting that website, EDGAR, to get all that information. We come second with number of hits, but we come first with a dispersion of all the things we're looking at, even the footnotes.

So you read the literature, you know that when a CEO is talking and if they talk too much, they're repetitive, I mean, there are red flags and stuff. So you can actually use natural language processing to associate certain terminologies and phrases in English to get inference, whether this is actually a good thing or a bad thing for the company. Analysts do that. They do it by talking, but you can do it with natural language processing. And it's the same techniques that Google invented in the public domain; in dot vec two, three, four, whatever it is. The other is an example that a colleague of mine, Michael Chen, and his colleagues worked on, which actually reads Chinese characters to try to get an insight into sentiment in the A-share market. And that's fascinating.

You can actually use natural language processing to associate certain terminologies and phrases in English to get inference, whether this is actually a good thing or a bad thing for the company. Analysts do that. They do it by talking, but you can do it with natural language processing.

Patrick: [00:31:19] Fascinating.

Eric: [00:31:20] As an aside, when I was at Solomon Brothers, of course we had a quantitative group which grew quite large that I organized. But we had a pretty good fundamental research team, who worked with sales for institutional investors and also with the investment banking people. And at one point, one of the things you needed to do as an analyst - that was understood - is you needed to detect deception by the CEO, CFO and so on. So we actually had psychologists come in and give lessons on how to interview and pick up things that might be red flags or green. So NLP is doing that, like all quant, it's doing it maybe not as deep, it's doing it systematically and broadly, that's the breadth and scale set.

Patrick: [00:31:56] Yeah. So one of the things we talk a lot about with our team, with other managers as well, is that in this world asking good questions, and you've kind of put this a couple different ways, has really become the creativity, I guess, is your word for it - has become the powerful thing, the differentiating thing. Because answering questions is getting easier and easier with more and more data, more and more techniques, et cetera. So as you think about your research agenda, how do you cultivate that creativity? Is it more of a bottom-up type system where it's just about hiring creative people and letting them be? Is there a top-down component to that? How is the research agenda structured?

Eric: [00:32:30] A lot of is who you hire and how you develop a business model, or if you will, a recruitment model and a leadership model. And people have come in that are creative, that are potentially asking the right questions. A very good friend of mine who is a contemporary academic came and visited me and visited our firm for a while after I'd been at Solomon Brothers, he said, "Eric, since you've been here, you become a much better academic." It was real. So I think, (a) trying to identify creative people, but, (b) giving them some rope, to a fault. I've always done this. Like with example of being an instructor pilot. I believe my best approach to managing creative activities and research and money management, in a business that has to make money for clients, is to allow people a lot of freedom, within reason.

And I think we have two CIOs in there, and I worked with these people for 20 years, and they've hired people and our equity group now have interns come in, and several of the group of the last three or four years have become full-time employees, they then become PMs. You can't over-control, nor can you come up with a list of like principles. You just have to enjoy it. I said earlier about the motivations.

Patrick: [00:33:40] The two things, yeah.

Eric: [00:33:41] Yeah. I think about this and I think about discovery. It's a good word. That's why I became an academic. And I had no idea that I'd ever be doing this today.

Patrick: [00:33:49] You figure out how stuff works.

Eric: [00:33:50] We're getting paid by people to manage their money, or in Wall Street, getting paid by people to trade their portfolios as a firm. But discovery is fun. One of my analysts, young guy, smart guy does our NLP stuff. You can't stop him talking. He came to my office yesterday. He says, "I got to come back and show the entire agenda for all this stuff." I would say, "I don't know what he's talking about." But I know enough about what he's talking about. If I don't want to make sure I get it, but that you got to understand.

Now he's not going to be the type of person that's going to sell, manage accounts, go out and pester, pester, pester institutions finally get into a semifinal, final and get business and then onload it, right? But you need that. So it's a fun thing to watch. And we have an environment where that is the way people do it. The other D is dollars I mentioned earlier, converting copy, people come into the business who they think you can make money, which you can, if that's your sole motivation that ultimately will not help you run an organization. So the right balance between having an economic setup, that makes sense - competitive - and also allowing some freedom of discovery.

Patrick: [00:34:50] Discovery in dollars is a good framework for thinking about that.

Eric: [00:34:52] I mean there are institutes that sponsor medical research that bring in young doctors, physicians, who are just doing research on incurable diseases today, Alzheimer's or some cancer. Here's money. Here's your team.

Patrick: [00:35:06] See you later.

Eric: [00:35:07] We'll see you later. And they're doing it because they just love to discover stuff.

Patrick: [00:35:12] One of the things I always like to ask about is this notion of a research graveyard. So everyone's got things that they were excited, hypotheses, that they wanted to test, that they went through the rigmarole of getting the data and testing and just didn't work. It failed. And it gets its tombstone in the graveyard. So I'm curious if you have any sort of favorite tombstones, things that you've tried or your team has tried that you thought were good hypotheses, but just didn't work?

Eric: [00:35:34] Oh, so many of them.

Patrick: [00:35:35] Yep.

Eric: [00:35:36] I'm trying to go back and stir the ashes of my own experience. For example, I think it's the case that this work we did recently, my colleagues did on the A-share market. He did his analysis of reading 50,000 Chinese characters. And they're not sentences. And they did an association between the language used in the chat rooms, there are three or four of these blogs, it's a retail-driven market in China and positive-negative sentiment. And because China, the country itself is kind of heavily regulated. And there's some censorship of what you can say, two Chinese characters, the two Chinese expressions, one is garbage or bad and the other spicy chicken. So you can say this restaurant sell spicy chicken, or this restaurant is garbage, you sound very similar. And you read these blogs and you can associate that. So the slang is, if you don't like it, it's spicy chicken.

Well, you can do unsupervised machine learning to actually come up with a whole set of those things. I don't believe we make any money on that, but I do believe this, we learned a lot, my colleagues did. And maybe ultimately that particular application will be useful. Is it in the graveyard? No. But there are other things I suppose. I mean, volatility with all the work at Solomon - metrics on forecasting volatility was a hot topic. And we had Rob Engle who became a Nobel Prize Laureate, Rob Engle from University of San Diego, UCSD, and spent a year with us. And we had a lot of fun with this. I kind of knew that we're not going to make any money on it, but it was really engaging.

So we use ARCH and GARCH models and we wrote some stuff about that. And went out on the road and because Mezrich, I'll mention him again, I got to Solomon in '87 and he was in another group - technology area somewhere - and had a PhD in math psych. and he was a time series statistician. Everybody's doing cross-sectional, cross-sectional even crudely at that. I couldn't find anybody who knew what an ARIMA was or any classic econometric time series. He did. I asked him to join our group and the rest is history. And I think - so we were doing this volatility modeling. I don't know if you make money on that. I don't know. Markets are pretty efficient, and implied vols compared with statistically predicted volume.

Patrick: [00:37:44] What about the state today of what classically is called risk premia? So if you look at PanAgora's sort of lineup, if you will, of strategies, there are two separate categories, both systematic and quant. One risk premium, one called active. So I'd love to understand the difference between those two, starting with what your interpretation of this idea of risk premia strategies looks like. And sort of any comments on the state of those strategies today after what you said has been in most cases, a tough couple of years for kind of classic quant models.

Eric: [00:38:10] It has been on the equity side, no question. With respect to risk premium. We probably think of it more as asset classes. It could be currencies. It could be bond markets around various maturity structures and bond markets, certainly equity markets. And risk premium means, if I'm going to basically expose myself to a volatile asset, I should get a return and you can measure the sharpe ratio of that return. And I said earlier, sometimes it mean reverts but Eddy Qian, who was the first one to actually write down in a paper in 2006 - he's one of our CIOs, who runs a great product area, risk parity. And Bridgewater had a product at the time, phenomenal name, because as a pilot-

Patrick: [00:38:50] All Weather.

Eric: [00:38:50] All Weather, man, we were always talking about the All Weather aircraft that was coming. Maybe it was the F-15 that could fly in any condition, by the way, you don't fly in a thunderstorm. So there's no such thing as an all-weather airplane or hailstorms, you just don't do it.

Patrick: [00:39:02] Great analogy there.

Eric: [00:39:03] But the formal way of doing that is to risk weight the assets. And so we have a product in that area. I think that has real staying power. What happens is sometimes - the proposition is that they all have similar sharpe ratios over time, but they go through different cycles. Really the correlations between the asset classes if they're low or negative, it's a winning strategy. You can accumulate more wealth for your client with a risk-weighted approach over a reasonable horizon than say a traditional 60/40. But it doesn't always work because - work meaning the following: it's underperforming traditional. What do you mean underperforming? Well, last quarter it was 3% behind the say 60/40, which is 90%, 95% equity risk. Why? Because equities went straight up.

Patrick: [00:39:42] Right.

Eric: [00:39:43] So I tell this to him. I said, you're doing your job when it "under-performs" because over three-year period, it'll deliver a better answer.

I said, you're doing your job when it "under-performs" because over three-year period, it'll deliver a better answer.

Patrick: [00:39:48] Yeah.

Eric: [00:39:49] And he went silent.

Patrick: [00:39:49] If you're not disappointed, you're not diversified, right?

Eric: [00:39:51] No, I know. And so we do that in long-only space. We also do it in, if you will, hedged or, if you will, beta or market neutral space. Same concept - capturing premium but offsetting it by shorting something else.

Patrick: [00:40:04] So active equity is a very different idea. So talk through, from what I can glean on the materials that I saw, it sounds as though maybe we can use the biotech or some other examples. There are sometimes even single stock specific models that are being built. So it's not long/short decile spreads and value or something. It's more granular than that. So describe that process, that active equity process.

Eric: [00:40:25] We have two approaches that we've used and actually they're two different portfolio offerings. One of them does, I never did like it, but we always talked about it - every stock has its own model. See, there's so much idiosyncratic risk in a single stock I don't think you can model it with valuation, but in terms of actually weighting factors, it's hard. But we built this process back - my colleagues and I was part of it - 20 years ago. And we have clients in this strategy today, we call it dynamic equity, which actually rather than one size fits all, the weightings on a particular stock in this set of factors, whatever they are: value, momentum, growth with different inputs over time - quality - the weights are different for different securities. No two securities with exactly the same weight. But the way the weights are determined is by commonality within a larger group of securities. So take all the growth stocks, the high growers, or take all the small stocks or all the high beta stocks and model with your inputs, what is the optimal weighting scheme for each of those clumps? Well, no stock is in one clump and Euclidean space is sitting out there somewhere. So you map it into these four or five different risk, if you will, partitions and then use a weighting scheme, which is drawing from where are the stock actually resides. That was pretty good. And even today that's still better than one size fits all, but other people have actually done that. I've copied this to some degree. So that's a one size fits all kind of away from that.

The other strategy gets very deep into the mechanics within a sector or when a set of common securities, what are the fundamental drivers...

Patrick: [00:41:54] Like the biotech example.

Eric: [00:41:54] Like the biotech.

Patrick: [00:41:55] Is there another example that you think is interesting working?

Eric: [00:41:57] Yeah. And I'll use an old example too, financials. They really had a hard time, quants did with their financial sectors of their portfolio in 2006/'7/'8. And it was a problem. And my colleague George Mussalli, who's the other CIO, the counterpart to Eddy Qian, who I mentioned earlier is really a pioneer in what's today called big data. I mean, it's a real fundamentally-oriented team, but they identify and they were trying to figure out ways to really sort between good and bad banks - financials - state and corporate, commercial banks. And one of the data sources they thought about, "Well, let's value the asset in the portfolio. Let's figure out what's the quality of the assets - all of the loans in the portfolio. Categorically, they're all kinds of loans." And so they figured because we know the world, well, there's a government entity that actually cares about that.

The Federal Deposit Insurance Corporation, because they guarantee to some degree the solvency on behalf of the depositors. Well, they know a lot about the quality of those loans. They could have their data and turns out they had at that time, I'm sure they have better data now, but they had data on the delinquency rates of a loan portfolio of commercial banks that were FDIC insured, meaning what percent of the loans were one month overdue, two months, three months overdue. So you could actually, if you knew that, give a haircut to the value of the assets and therefore value the bank. By the way, you can also do with insurance companies, you can also have real estate and REITs. We do, we do it all over now.

But here's a great evolution of a factor. So George and group, they call the FDIC, can we have that data? They had found out about it. Sure. And I like to describe it this way. It probably wasn't quite literally, but about a week later, a huge box of paper arrived at our office. All this stuff typed up, written down or printed out. We can't use that. The labor going through. Can you get it to us electronically? And they couldn't, but they ultimately were able to.

So going into 2007 and '8, I believe our financials performance, as part of our long-only stock selector product actually held up a lot better than most of them. Now, that same data today has been arbitraged away because I'm sure that one or two clever providers of what's called big data found out about it, went into the FDIC and said you've given that to PanAgora. We'll buy it.

Now, that same data today has been arbitraged away because I'm sure that one or two clever providers of what's called big data found out about it, went into the FDIC and said you've given that to PanAgora. We'll buy it.

Patrick: [00:44:04] Yeah.

Eric: [00:44:04] Okay. And then they'll resell it. That's a stylistic version of what I think is a good example.

Patrick: [00:44:10] Let's take those two models. Again, how often are those models being updated or changing?

Eric: [00:44:14] Well, a factor may last four or five years and the half-life may be two years and you have to monitor it, how well it's working. We have a model that our equity team use, a model in quotes and actual expected value model or a business model of whether we're going to go after a particular dataset and then use it in production, big data, smart data, whatever. Part of the answer is, how much does it cost? The other part of the answer is we look at the existing model, will it add value at the margin? So you can get a set of data that's a, if you will, a version of it, a pilot version, you can test it with your process and see - you can get a pretty good answer. Will it add three basis points or four? We think we have to guess a little bit on the bound of that.

And then generally say, what's going to cost us? Some of this big data is very expensive because the hedge fund world is a smart people. They get in there in a hurry and it bids the price up. And so then we have to make a choice. Are we going to buy it when it's not economically worthwhile, or do we buy some? Or, are we going to try an alternative source? Or are we just not going to do it?

And by the way, again, this team, we have a medical doctor on there. We have other people from different parts, a historical professional, the MD didn't practice medicine, he's been with us 10 years. He built our biotech and pharma model. He loves this stuff. He just didn't want to be a doctor, but he's in the field. We have a way of actually sorting through what will work and what won't work. And will work is, if the vendor says this is going to cost you $200,000 a year and you can find - let's say you want to know, there's a building right over here, you want to know the occupants. You think you have some big data or some smart data that can figure out the quality - the staying power - of the occupants. So the real estate holding the REIT, whoever owns it, you can rank whether that's safe, risky, or not.

Patrick: [00:45:56] Underwrite the quality.

Eric: [00:45:57] Right. You can buy that data. That's very expensive. Or you can go down the street and find a person who's basically washing windows or a person that's actually in the plumbing business or fixture business. And there are 40 tenants in that building and that guy's got access, or that person has access to information on who the tenants are. So they know who to go after. And the trade organizations have some of this, it's free. So that's not always the case.

Exciting Frontiers of Research

Patrick: [00:46:19] It's a fascinating kind of application of some of the Porter forces to business where the business in this case is supplier power or whatever it is. The inputs into the quant process are complicated. And like you said, the discovery is, what's so fun about this, is relating sort of raw data sets to outcomes that we care about for our investors. As you look forward today, what are the frontiers of research that excite you most sitting on top of so many different strategies, new techniques, new data sets. Is there any one or two things that you're really interested in today, or is it just sort of the same perpetual discovery that is driving you?

Eric: [00:46:53] On the multi-asset side, whether it's broad markets, equity markets, or sectors I think there's a lot of room for better time-series work. Again, I mentioned Mezrich, a lot of people are focused in on cross-sectional comparisons and don't have a real passion for like, over long cycles, can we forecast better? Or can we be on the correct side of it? That's a whole area that I think has got a future.

In equities per-se, you look what's happened. We have quant equity. It's under some pressure. It's like fundamental. Basically, there's not a lot of flow or the flow is slowed down. It will pick up again. For the ranking system. I like to think of the ranking system of ranking stocks, I often compare it to the ancient Olympics - it's 1776 BC, the Greeks, they had a stadium and they started a contest, who could run around this 200-meter stadium, the fastest. They had a contest.

And then over time, over the next 50 years, they proliferated the event, not with the number of people participating - yeah, people could participate - but with how many events that person had to participate in. It became the pentathlon. There was running, javelin throwing, discuss, jumping, and wrestling. I think about all those things. Think about a stock, five characteristics, is it going to be at the top on every one? No, the winner of the pentathlon on a weighted average basis, won. You weigh these events. I think Jim Thorpe from the 1912 Olympics is still viewed by many as the greatest athlete of all time. He could do them all. He could have won the individual events.

That's traditional quant equity. We want to rank. And we get in there and do that. So that whole world is about data. And it's about the tools to actually forecast. We mentioned them, the machine learning or the AI type things as new ways, which are basically extensions of linear regression or these data sets you and I have talked a little bit about. And you just have to hope and think that you can actually be ahead of the field in that.

Patrick: [00:48:53] I'm curious if you think there are any safe havens for non-quantitative investors. So like Marc Andreessen always says, software's eating the world. Kind of think about quant the same way. Like if quant can eat something in investing, it's going to. We can do whatever an analyst does more efficiently if it's systematic and structured, do you think there are safe havens for the traditional, more old school discretionary analyst or portfolio manager?

Eric: [00:49:15] We go to see clients, there's this group I'm thinking of. They usually more sophisticated clients. They like us. They like our competitors. They have multiple of us. First question, head guy in the room, person in the room, what are you doing that's different. Be good, but be different.

Patrick: [00:49:28] Yeah.

Eric: [00:49:29] I think concentrated portfolios - forget this alpha, beta separation. That's going away. Smart beta as we call it, you can't have a monolithic cap-weighted MSCI that everybody has a track to. It's going to change. It is changing. And I'll come back to another metaphor about the Olympics of how I think that's working its way out. Modern Olympics, as opposed to the pentathlon. And therefore those that actually can pick stocks and have concentrated portfolios, I think there's a whole world for them. There's a key though, you can't hamstring them. You can't make them have a beta one. In fact, you want to say, I like your team. I like your processes. And by the way, that's a large part of how you get hired. Not just your track records. I believe you believe what you're doing. I like your passion. I like your fees, but that's not the main thing, we think this will work. And by the way, if it doesn't, we'll fire you. By the way, we as money managers, exist so we can be terminated. It's really is an agent-principle problem. It's not a bad place to be as long as you can survive.

So concentrated portfolios. And one of the keys is, man, can I hold cash some time? Yeah, you can do that. See, in 1971/'72 I went back to graduate school. And I started looking at this. Every asset manager was alternative, the history of this company, probably with your name on it was alternative. It wasn't viewed as, okay, here's a mandate, look like the S&P 500 or look like the NIFTY 50.

Patrick: [00:50:46] Right.

Eric: [00:50:47] So we're kind of moving back to that, but with better tools, I hope we're moving back to that where the goal is wealth creation for the ultimate owner or sharpe ratio, periodically, but it's ultimate wealth creation. So does quant equity or does fundamental survive? I think they both survive.

We're kind of moving back to that, but with better tools, I hope we're moving back to that where the goal is wealth creation for the ultimate owner or sharpe ratio, periodically, but it's ultimate wealth creation. So does quant equity or does fundamental survive? I think they both survive.

Patrick: [00:51:03] What's your modern Olympic analogy.

Eric: [00:51:05] Well, I think about what's happened in smart beta, it's a metaphor. If you look today, let's say you're going to field an Olympic team today. We're well beyond just the pentathlon. It's a single activity. In fact, people look at that and say, that's interesting, but they really focus in on the specialization; the sprinters as a group, the hammer throwers as a group, the high hurdlers. And so when an entity is going to put together an Olympic team today, it will bring clumps or specialists that have one thing they do well. You don't make the 165-pound sprinter throw a javelin or the 300-pound shot put player sprint. So what's smart beta? Smart beta is sort of a decoupling of alpha and beta, but it's actually getting the portfolio exposure to these factors, to these events with clusters of, if you will, specialists.

So we have a defensive equity strategy. We can run a low vol one. We can run a dividend yield one, which we do. We can run a multi-factor one. And this portfolio construction technique, which trades off diversification benefits of the stocks and the stocks' specialty. So it's a little bit of simplistic, but I think that stuff, it's all over. It's not done correctly every place. There are lots of concentrations of risks and the way it's been done, historically, you may end up with a whole bunch of utility stocks for low vol, but there are better ways to construct the portfolio. And I think there's some room for growth there too.

Advice for Young Quants

Patrick: [00:52:16] You mentioned earlier the example of the guy who won't stop talking he's so excited about research in your office. I'm curious if there's any advice you have for young, let's call them quants interested in getting into quant finance as practitioners. Any advice that you would give that batch of people today in terms of things to focus on, to differentiate themselves tools to avail themselves with, et cetera.

Eric: [00:52:36] Again, my team and including the gentleman you just referred to, we look at machine learning in a certain way and it can be applied all over - a different engineering problem or driverless cars problem. And there are, I don't know if it's unique to my colleagues, but it's a nice way to think about it, there's a Venn diagram with three circles, there's an intersection called machine learning. One of those circles is domain knowledge and another circle is the analytics, the algorithms - that computational expertise or acumen - and there's data.

So if someone wants to come into this business or any business, they want to go in, they got to understand all of those three are important. So I have an acquaintance who runs a big money management firm, he asked me, "You've got really good quants. We go to the universities and our people will look at the advanced degrees in statistics. They don't all work out." Well, they've got that one circled, they know the techniques. Maybe don't understand the industry really. And certainly they don't know the quality of the data yet. They have to learn that.

So I would say understanding that there's data, that there are analytics, which you learning in school a lot of it, and the data may not be good data to use it because it's commercially available and universities can afford it. And there is, do you really know what a trader does? What makes prices move day-to-day? Do you understand dealers? Do you understand buy-side clients? Do you understand stickiness, you understand whatever cycles?

So you're not gonna learn that overnight, but just be aware there are those other components. So get exposure. Again, I mentioned my colleague, George Mussalli, who he tells his story. He was an undergrad, and then he went to MIT ultimately, but he had to decide whether there's going to be a fundamental analyst or a quant. I think he's a good quant because he spent a lot of time thinking about what a fundamental analyst has to do. So you think about domain knowledge and also data.

Patrick: [00:54:14] I love that domain piece as a recommendation for young aspiring analysts, it is so incredibly important because it's like a search cost producer for the manager that's going to hire you. It's just such a hard part of the process that if you can bring that to the table, that's a huge relative advantage, competitive advantage for a person.

Eric: [00:54:31] When in doubt just hire a jet pilot. Then you're done.

Patrick: [00:54:34] Good to go. Last couple of questions. So the first is you mentioned Joe Mezrich as an example of kind of what I'm asking-

Eric: [00:54:40] Is it okay I mentioned people in these podcast?

Patrick: [00:54:41] Oh, yeah, absolutely.

Eric: [00:54:42] Okay.

Patrick: [00:54:42] Encouraged. We want this to be a tool for people to go do their own further research. So my question is specifically on the buy-side, if there are other quants that you especially admire, who you think do really good work, who are worth reading their work, following, et cetera?

Eric: [00:54:55] Probably. I don't even know who they are. I'm not necessarily sure it's people that are actually doing it for money. I think there are people that are doing it in other forums, maybe research groups. We have a hard time competing now. I mean, finding talent because they go to Google or they can go to Amazon and there are other - in Boston or other towns like Montreal - there's huge work in AI, there's machine learning. And what used to be an outlet for those, that talent pool has now got more possibilities.

I can name some people but they're my competitors and I don't want to.

Patrick: [00:55:23] Good answer. Usually, I can trick people with that one. You didn't fall for it.

Eric: [00:55:29] No, actually I'm probably not in touch as much as if you asked that question to the CIOs I've mentioned earlier, I think they would have answers.

Patrick: [00:55:37] Yeah.

Eric: [00:55:37] Because they're hanging out more with that group than I am at this point.

Patrick: [00:55:41] So my closing question for everybody is for the kindest thing that anyone's ever done for you.

Eric: [00:55:44] Really? Oh man, loved me unconditionally. That's a big term. That's a religious thing.

Patrick: [00:55:50] Yeah.

Eric: [00:55:50] Love me unconditionally. Beyond any notion. No doubts.

Patrick: [00:55:54] Wonderful.

Eric: [00:55:54] I'm good to go. I'm now free from guilt.

Patrick: [00:55:56] That's fantastic. A unique answer. I've done 135 of these. Haven't gotten something quite that direct and pure.

Eric: [00:56:02] And that's the truth.

Patrick: [00:56:02] So thank you.

Eric: [00:56:03] Can tell you more about that.

Patrick: [00:56:03] Great. Well, thank you so much for being here. I've learned a ton.

Eric: [00:56:05] Well, Patrick, it was great. Thanks a lot.

Created by @domcooke