My first job out of university was a plant engineer at an underground gold mine. I reported for duty thinking I’d be crunching numbers and designing stuff. Instead, I was told that for the next six months, I would be an assistant to this grizzled old mechanical fitter who worked on keeping the underground plant running. What I gained from those six months of crawling around in the dust, the grease, and the dark with that man has proven just as valuable (perhaps more) as the four years I spent in lecture halls.
He knew, on any given morning, which of the dozens of things that were broken or about to break underground actually mattered that day, and which could wait. He knew that because he knew where the drillers were working that week and what the loader operators had been grumbling about. He could hear a bearing on its way out from the other end of a drive, which was impressive, but the sound was only ever a clue about this machine, right now. What counted was what he did with it. He knew where a bandaid would do and where it wouldn’t.
The drillers and loader operators were his information network, though nobody down there would have called it that. The relationships were as rough as you’d expect, but the blokes told him things, in the crib room or shouted over the top of a loader. Most of what he knew about what mattered that day arrived that way. A fresh graduate engineer could only dream of building that kind of standing.
So when my boss, who sat in an office on the surface with the maintenance plan, told him to do one thing, he’d quite often go and do something else instead, without consulting anybody, because something had come up that mattered more. To his credit, my boss almost never questioned this.
I learned some rudimentary technical skills during that six months. Obviously I couldn’t have hoped to emulate what this bloke had learned over an entire career, but I can fix a pump when I need to (our house is plumbed with rainwater we collect on the property, so occasionally something around here needs fixing).
More than the technical skills themselves, I learned something about the skills. There’s a category of knowing that doesn’t come from book learning or listening to lectures. In the past, I probably assumed that was because it was too subtle to teach and could only come from experience, from actually doing the thing. But I think it’s simpler than that. It’s that this sort of knowledge only exists where the work is being done, and it’s perishable. Which of the many maintenance issues matters today is useful information today and useless tomorrow.
I’ve been doing some reading to try and figure out what I think about all this, and in the process came across Hayek’s knowledge of the particular circumstances of time and place (he was writing about central planning, but I’m not going to let that stop me quoting him in an article about gold mines and systematic trading). According to Hayek, there’s the general kind of knowledge, the theory of bearings and gears for example, which can be written down and kept in one place. And there’s the local, perishable kind, which only the person on the ground has, and which is of any use only if the decision is left with them. He even makes the point about apprenticeships in passing, something to the effect that we have a great deal to learn in any occupation after the theoretical training is done, and that a big part of working life goes on learning particular jobs. He wrote that in 1945. The fitter would have been a massive fan of this idea, and wouldn’t have been backward in telling me all about it. I think I’m finally getting it too.
It occurs to me that that’s what his lone-wolf-style rescheduling was about. My boss let it slide because he couldn’t make that call from an office on the surface (and he was smart enough and humble enough to realise it). The information needed to make it was down the hole and it was down there right now, and by the time it reached his desk it would be a different set of problems. So the decision had to live with the bloke who had the knowledge.
I’m not saying my engineering degree was wasted (although to be fair, I was, a lot, during those years). The classroom learning provided an important framework that turned that master fitter’s calls in the field into transferable judgment rather than habits to be blindly copied. It was a set of hooks onto which I could hang the practical chunks of wisdom I was receiving 600 metres underground. He knew enough about the theory of gears and bearings too, of course, but what he also had was the ability to interpret what was in front of him right then - the sound of the machine and the mood of the drillers - and decide what to do about it.
This is not a takedown of classroom knowledge or a glorification of practical knowledge. The two kinds need each other. The difference is that one can be centralised and the other can’t. You can put the theory of bearings in a textbook... and now you can put it in a language model. You can’t put “the pump on level twelve can hang on for another day” anywhere useful, because by the time you’ve written it down it’s stopped being true, and the only way to act on it is to leave the decision with whoever is on the spot.
Cut to today. Most of what I build or fix in my professional work is digital. And the cost of building digital things - in terms of both time and money - has suddenly collapsed. That means that I have the luxury of spending nearly all of my time operating at the level of a PM, lead engineer, or head trader (I have a strange mix of business interests). That means thinking about what questions are worth asking, what gets built, and why.
When I first started doing low-latency trading in crypto, for instance, the software development side of things was the bottleneck. Even using components we bought off the shelf, the ideas could only be implemented as fast as I could type and debug (which is not very fast). Today, the ideas can be implemented as fast as I can strategise what to build and review what gets built. I expect that as AI improves, the review side will shrink dramatically too. The jury is still out on the “strategise” stuff, but I don’t think the verdict matters much.
Note that I deliberately didn’t say “the ideas can be implemented as fast as AI can code.” Even though the AI is doing the vast majority of the coding, its speed is not the limiting factor. The bottleneck is still me. But it’s my capacity to make decisions that sets the pace at which we move, not the speed at which I write code.
What I spend most of my time on has shifted from doing to thinking about what to do.
In the terms I’ve been using, I’ve moved into my old boss’s office. Upstairs, on the surface, with the plan. I can get away with that because I did my six months down the hole and a good part of the twenty-odd years since with real risk of some sort on the line. Those risks are still running, so the noises from the pumps still reach me. The person I’m worried about is the one who gets the office without ever having been down the hole.
My sense is that there are a lot of people feeling empowered to build stuff right now. And fair enough. The value of the process of doing the building collapsed. So why not build?
Because, maybe, the thing you want to build just isn’t worth building.
I know this because I was that person once. When I first got interested in trading, I built machine learning frameworks for it, elaborate ones, and you can read about them in embarrassing detail in my case study. I was proud of them. Then I showed one to an actual trader, who asked me what my edge was. I didn’t have an answer. I’d built the thing I assumed I needed without asking the question that underpins it all. Turns out I could have used some of that practical knowledge. The person who’d actually spent time doing the work could diagnose what I couldn’t see at a glance.
These days I’m on the other side of that exchange, several times a week, via my inbox.
Earnest entrepreneurs who’ve built something that “my community will find incredibly useful”, if only I were willing to test it out for a week or two. Sentiment widgets. AI stock pickers. And the most common of all, an application that can tell you if your backtest is “real” using some sort of statistical test.
Most of them do things nobody asked for, and nobody would pay for. (Worth building instead of that last one: a piece of research that helps you figure out if the underlying effect is real—save the backtest for what it’s actually good for.)
And they got built for the same reason my frameworks got built. Plenty of general knowledge, nobody on the spot with practical experience.
This probably reads as a dig at the well-meaning people who send me their vibe-coded apps, and on some level it is (yes, it’s starting to wear thin... I might just be the grizzled fitter now, and he had opinions about engineers fresh out of university too). But these people didn’t skip the apprenticeship out of laziness or ignorance. For a lot of them there wasn’t one to skip.
If the work of building is now cheap and doesn’t require a person, there’s a layer of the org chart that must be getting thinned out, and the people there have to go somewhere. I’ll hedge that a bit, because tools that made building cheaper have so far produced more builders rather than fewer, and that could very well happen again. I want to be very careful about being one of those people who say “this time is different!” I’m more confident that the grunt-work part of the junior job, the part that got you a seat next to someone who knew things, is the part being automated.
Inside a firm, my guess is that that layer gets removed altogether, and the junior never gets hired at all, because the work that justified hiring them is done by a machine.
The independent trader never had a fitter to offside for in the first place. If you came to trading at forty from engineering or data science, you had the degree and you went straight to the office upstairs, because there was no desk to sit at and nobody around who’d lost money and would tell you about it (I now realise how lucky I was to stumble across this missing piece). The graduate is in charge of all the other graduates. There’s a lot of energy and a lot of knowledge and not much experience in that specific domain. Building was the one thing you could always do for yourself, and building is now close to free. What’s left over is the thing you never had.
I explored this idea in “AI Will Create Millions of Quants,” which suggested that AI will fool beginner traders (where a “beginner” is someone who hasn’t done their apprenticeship) into thinking that they’re doing the real work that really moves the needle in a trading operation, when instead they’re just making all the fresh graduate mistakes faster than ever without realising they’re even making mistakes. A lot of what I see here in the quant space on Substack suggests it’s coming true.
So why doesn’t the model just fill the gap? It’s read every textbook on the topic and a great deal more.
I’m least sure about my answers to this because it feels like they should be at least partly technical, and I’m no LLM expert. But I can see how the model changes the incentive structure for humans in ways that have unintended consequences.
Firstly, the friction of building was actually a really good filter. When building took three months of your time, you had to convince someone (a boss, yourself, other people competing for your time). That friction did quality control on ideas. Of course it wasn’t perfect. Lots of crappy products still got built (I built lots of them myself). But it was real friction and a real filter. Even if some crappy idea got through that filter, and you spent three months building something that should never have been built, you would have finally realised that you’d wasted your time. And your wasted effort was motivation to ask “what did I get wrong here?” And you would learn something important. Most of that has gone.
I’ve floated this idea with a few people in real life, and a common pushback is that cheap building means you can build lots of things quickly and learn what’s worth building by seeing what sticks. I don’t think so, because these are not real reps. They’re like the data mining approach to trading—try enough stuff and something is bound to work, right? Maybe, but unlikely. And even if it did, you’d never be confident in it because you don’t know why it works in the first place. That’s why I’m always banging on about mechanism-first thinking in trading.
A real rep—something that teaches you something or improves you in some way—is a decision you made and had to wear the consequences of, with somebody in the loop who could tell you where you went wrong (or some other kind of useful feedback mechanism). Fifty things built from the office upstairs are fifty reps upstairs. None of them tell you which issue actually mattered on the ground.
This seems to be true in general technical and knowledge work. But in trading it’s way worse, because the feedback mechanism you assume will set you straight, the market, is slow and noisy. You can be right and lose money and be wrong and make money. Happens all the time. The consequences don’t explain themselves. You need an apprenticeship just to come to terms with what you can and can’t learn from the market itself! Chess engines made human players better because the feedback in chess is instant and unambiguous. Markets are the opposite of that.
So the reps have to be real ones, down the hole, and they have to happen next to someone who can look at them and tell you where you went wrong.
In addition, the model is a repository of general knowledge. Every textbook on bearings, every paper on momentum, every forum thread where somebody explained why you have to use a stop loss to manage your risk. It’s the best-read graduate in history. I mean that mostly as a compliment, because that kind of knowledge matters, except the stuff about stop losses of course. But Hayek’s point was that this was never the hard part, at least in terms of making good things happen on the ground. The scientific, textbook kind of knowledge was always the kind you could centralise. The hard stuff is the other kind; it’s more nuanced, is based on judgment, and it doesn’t get into the training data. That’s got nothing to do with how clever the model is. It was never written down, and it’s stale anyway because it’s about what’s happening here and now.
Is there a second-order effect here too? The people who write things down on the internet probably skew towards theorising about the work rather than doing it, so might even the written record tilt away from judgment?
So the model is the office upstairs. It’s got a plan. It’s highly educated. But it can’t tell you which particular job matters today because it doesn’t have the experiential context. More training won’t help, because the missing pieces are not in the training data. The fitter I worked with could override my boss because he could see something the boss couldn’t. The model can’t see anything you haven’t explicitly shown it. If you ask it whether an idea is bad, it’ll have a crack, and sometimes it’ll be right, but the feedback carries no real weight. If I, as a fresh graduate, offered an opinion to the fitter who’d done the job for 20 years, he’d have listened politely and pointed out all the things I hadn’t considered. The gap was not my education; it was my hands-on experience. And someone with the right experience knew exactly where to push back. And so it goes with LLMs—you can ask it for help on a real-world problem, but if you don’t have the real-world experience, you can’t judge where the holes are in its answer, or even recognize that there are holes in the first place. It’s like two fresh university graduates trying to figure out what the master fitter would do when faced with a problem 600 metres underground.
The model builds whatever’s in your head. And that’s the problem.
When I’ve floated this idea with people they nearly always say “well then give it the time-and-place stuff”. I do this constantly, and it helps, but it still doesn’t get you there, for a reason that I’m only just now figuring out how to articulate. Knowing what to put in the prompt is the judgment. The fitter I worked alongside had noticed a hundred things in the first hour of the shift, had dozens of conversations in the smoko room or just in passing, and he knew which ones carried information that needed to be acted upon. When the loader operator’s grumble went straight into the pile marked “these matter,” it did so without him even noticing he’d done any sorting.
The point is that you can’t feed the model the thing you didn’t register as information! So what you end up getting back is a confident statement about the part of the situation you thought to describe, which is a very good way to feel like you’ve got a master fitter in the loop when what you’ve actually got is a well-read version of yourself.
Now that I mention it, I can see places where I’ve fallen into this trap myself. Perhaps it’s inescapable.
So where do tomorrow’s fitters come from if the offsider job is gone?
I’m an optimist, and I think we’ll figure this out. But I am worried. The market will still deliver its consequences. What AI takes away is the structural reason for a junior to be sitting next to someone who knows things while those consequences materialise. Looking back, I can see that old farts like me essentially got that seat for free because somebody needed an offsider. The judgment I picked up in that seat is the sort of thing that’s very hard to even notice you’re missing. You don’t know what you don’t know, after all. I worry that the next people to come through will not only not know important things, but will have no inkling at all that these things even exist to be known.
For the independent trader, none of this should be news. If you never worked in the industry, you never had the seat. You had to somehow figure it out yourself. That gap is the reason the trading community I run exists at all, and it predates ChatGPT by almost a decade. What’s changed since ChatGPT et. al. came along is that the gap was once hidden behind the process of building. You could spend months or years writing a backtester, building machine learning frameworks, and simulating your way to the right equity curve, and you’d feel like you were serving an apprenticeship. Now you can do those things faster than ever, which I guess feels like ever-faster progress, but the thing that was missing all along is still missing. All that’s happened is that the hamster wheel to nowhere has had a flywheel bolted onto it. It feels like runaway acceleration. That’s what prompted me to write this.
I sell the degree, and I run a trading desk, so I’m about as unbiased on this as the fitter was on fresh university engineers. Discount accordingly.
The fitter can be a mentor, a job on a trading desk for a couple of years, or a room full of people who’ve been wrong in ways that cost them something. The desk I run is one of those rooms, and it’s a very good one. There are others, of course. But you have to do the apprenticeship, pay the fee in time and effort, because the thing you’re after only exists in that process, and you have to go and find it. That’s hard, mostly because it first requires the wherewithal to notice you need it, and who has that at the start? I certainly didn’t. Then there’s the sweet seduction of the AI—why take the slow road when AI will build whatever’s in your head right now (and tell you what a fantastic idea it is the whole time)?
One thing I’ve realised recently is to be OK being the bottleneck. You’re the one on the spot, whether you like it or not. The model is the office upstairs, and the decision was always going to have to be made down the hole by whoever can see the pump. I think we all feel an anxiety to do more in less time now that we have AI. But working on the wrong things over and over again at ever-faster speed is much worse than working on one good idea at the speed of your own thoughts. Don’t aspire to build at the speed AI codes at. Aspire to build at the speed you can come up with, test, and validate good ideas. The deciding is also where the reps are now, because it’s the one part of the job that still happens on the spot.
That’s all well and good, but you still need the judgment to come up with such ideas in the first place, and that’s the bit the model can’t do for you. It knows every paper on momentum ever written. It doesn’t know what’s in your book, what your broker charged you to borrow last week, what you noticed about your fills during early October (is it just me or did liquidity get a bit thinner across the board?), or the handful of things you’ve been expensively wrong about before. That’s your knowledge of time and place. It’s the stuff edges are made of, when you know where to look. Funnily enough, Hayek’s own examples of people who make a living from this kind of knowledge were a bloke filling half-empty tramp steamers and an arbitrageur.
The degree is important. I hope I haven’t given the impression of disparaging the value of classroom knowledge. It provides a frame—the hooks to hang things on. In contrast, the apprenticeship builds the judgment, by making you decide things on the spot, with consequences, ideally next to someone who’ll tell you why you got it wrong so that you develop judgment, over time. When you think about it, the actual building used to be how you paid for the apprenticeship. You got the seat beside the fitter because he needed an offsider. The AI does the building now, but I think it’s still possible to serve that apprenticeship. The next part of the article is about exactly that.
