0:00 AI is now in almost every major organization on Earth. It's solving math problems and generating hundreds of millions of dollars in revenue for businesses like you or I. There's a bit of nuance here. It's not actually the AI model itself doing any of that, because an AI model is just a brain in a jar. It can't actually talk or take any action. The real work is being done by something else, and that something else is called an AI harness. In this video, I'm going to teach you everything you need to know about them. So, let's start with something you guys are already familiar with, Claude Code and Codex. For those of you guys that don't know, Claude Code and Codex aren't AI models. What Claude Code and Codex are are harnesses around the AI models. And so, Claude Code is Anthropic's harness for the Claude family of models. That includes models like Mythos, Fable, Opus, Sonnet, and Haiku. Codex is the OpenAI harness around the ChatGPT series of models, like GPT-6, GPT-5, and so on and so forth. And so, I'm going to be explaining harnesses using Codex and Claude Code. But, I want you to know 1:00 that this applies to virtually all harnesses out there and all underlying AI models. The key things here is both of these harnesses allow AI to do actual work, not just talk. Interestingly, if you guys were to take Claude and then put it in an OpenAI Codex harness, it would perform pretty differently. And so, if this is our underlying brain in a jar, okay, and this is its performance on a task, and you put it in harness A, it'll actually perform quite differently than if you were to put it in harness B. And that's why understanding AI harnesses is so important. There are a lot of studies and results out there now that show the exact same model can get massive gains in performance if put in a slightly different harness. And there are also preferences between models and harnesses that I think are also important to understand. Now, a chat window, in and of itself, believe it or not, is also a harness. And most AI models came from simple chat windows. And most of the AI tools that you guys know of now initially originated in chat windows. Now, obviously, we've gotten a lot more complex, which is why I'm making this video. Finally, AI progress occurs on multiple different dimensions. 2:02 We have progress in the underlying model intelligences, but we also have progress in the harnesses as well. And so, when you see a new benchmark drop where, you know, Claude code crushes it on a capability that maybe it previously didn't have access to, it's not just that the underlying intelligence got smarter. The underlying intelligence could have remained the exact same, but the harness around it could have gotten better, which enables that intelligence to basically, you know, strap itself in like a StarCraft marine and do more work in less time. So, let's chat the model on its own. The model really is just a brain in a jar, right? The way that AI models, at least the ones that I'm talking about here, were first trained, called large language models, were as text predictors. And so, you would give it a little string of text in, then it would give you a slightly longer string of text out. The thing is, these had no memory, meaning every time you started a new chat, it would be completely empty. They also couldn't see anything. So, they didn't see any of the files in your computer or on your screen. They didn't have the ability to act. They could only 3:00 tell you what to do. And so, if you wanted help with something, it would say, "Okay, start here and do this, then do that." And that very well could have worked for most cases. But obviously, if we really want to be productive, we just give the model the ability to do it themselves, right? We didn't always have access to that. Finally, it can't check. So, it doesn't know if its code works or if its creations look good. There's no verification loop. Now, obviously, we've solved all of those, but back in the day, all models are really capable of doing is, you know, you fed it in a string like the sky is, and then it would feed you the same string just with an additional word or token, like the sky is blue. And so, that's all that large language models do under the hood. We feed them a tremendous amount of data on basically all of the text on the internet and text off the internet as well, many, many, many times, and we get them to the point where if I fed them a string like the sky is and said, "What's the next letter?" it'd be able to tell us, "Well, statistically speaking, you know, the word blue has come after the sky is like a bajillion times, whereas 3:56 the word pterodactyl has come after the sky is like 14 times. So, it's probably not pterodactyl. It's probably more likely to be blue. What the harness is is basically, if you were to just use the model alone, you know, the model is kind of like a it's like a horse, right? It has a lot of power, but there's no way to control it. Sometimes the text will come out very intelligent, other times it'll come out kind of weird. And so, these ups and downs and both intelligence and then the lack of a model's ability to do things in the real world mean you don't have any control. What a harness is is a harness is kind of like the the saddle. It's kind of like the reins, right? It's kind of like the infrastructure that you put around the horse in order to allow it to proceed in a direction that is meaningful or useful to you. Cuz you know, the horse itself is extraordinarily powerful, but you know, it's not really being used to human ends. The harness is how we take this extraordinarily powerful intelligence and then we direct it to, you know, what we consider to be useful goals. And so, there are as of the time of this recording five major parts to harness 4:51 that I'm going to be teaching you guys about. Now, we have the model, okay? And that's that brain in a jar. Surrounded by it is context, which discusses what you are doing and why. Memory, which details past conversations you've had with the model. Tools, which allow the model to take actions in the real world. Verification, which are mechanisms by which a model can determine, did I do the right thing? And finally, permissions, which is a stop condition that stops a model from doing things that you don't want it to do. And so, this is sort of like the saddle or the reins of your horse, where the horse here is a model. And at the end of the day, a good harness is now just as important as an actual good model. You can feed in a wonderful harness and give it to a really crappy model, and the results can actually be pretty good. So, let's start with context. Understanding context will help you build and design far better AI agents. You know, when a new session starts, the model has zero context on more or less what's going on. It's instantiated almost as like a blank slate. And so, if you were to say hello, I mean, it would probably say hello back, but it wouldn't say, "Hello Nick, 5:50 how's your, you know, business going today?" It would have zero context around who I am. What context really is, at least in business terms, which I think most people here are probably most interested in, is what project are you working on? What's the file structure? What sort of recent changes have you made to your file structure? And what sort of tools do you have access to? What we do with context is we basically give all this information to the model as the very beginning of our text conversation. And we basically just hide it from us, so it doesn't ever complicate our view. But we make sure the model has access to all the information that it might need to make a context relevant decision based off what we say next. And so, you know, the way that we do this in, you know, Codex and Claude Code, for instance, um usually involves a code map. It involves usage of files like Claude all caps {dot} md or agents all caps {dot} md, where you put information and recommendations in, like, "Hey, use our code style. Don't edit this folder. Always run these tests." and so on and so forth. Okay? But other models will have their own conventions. And other harnesses will 6:45 also have their own conventions. The cool part about context is it really just gives the model an environment to act in. And you only ever need to create it once. Because you only ever need to create it once, it saves you the time from having to continuously re-prompt. It's basically when you solve the context problem, you solve all future instances of your prompting problems. And so, that's why it makes a lot of sense to get your context right. And that's why so many AI harnesses try and get the context as structured and efficient as humanly possible. Next up, you have memory and sort of a subset of it called compaction. The longer you talk to a model, okay, the more of its context window gets filled up. And so, this is a byproduct of an of an AI model. This isn't a harness thing. Uh you know, models have a very fixed context window and that's just based off how they were built and how they were grown. Uh unfortunately, obviously, when we reach the context window, there are certain dynamics that change. The model's intelligence tends to go down a little bit. There also ends up being usually a lot of conflicting instructions because when you get your 7:39 context long enough, we're talking millions of tokens or 20 to 50 books worth of context. Uh there tends to be, you know, one instruction saying do this, another instruction that says do that, and models tend to get pretty confused. And so, what memory allows us to do is allows us to take everything, our goals, the plan, the files we change, various steps we're on, errors, all of the notes, all the failed attempts, and so on and so forth, and then turn that into a much more compressed compressed high-density representation of that data, so that the model can continue operating in the same direction that it was doing before, but it can do so, you know, smarter and without sacrificing a lot of the context that we previously had. Okay? And so, what ends up happening with compaction, which is a specific memory feature that Harnesses do really well, this is really the core of AI Harness memory, is we tend to summarize everything into a much more condensed version. Okay? Old steps might become one short summary. We might remove the attempts that failed and just detail what's going on. And then, as a result, free up the context window to allow models to talk for much much 8:38 longer than just their initial context window. And so, Claude Code, for instance, will do this automatically when the window is almost full. In addition, there are a couple of additional memory features that are quite useful that most Harnesses will do. You know, if you work on the same file multiple times, what the Harnesses will do now is they'll usually create a change log of your changes. They don't actually store the whole file itself in the memory. What they'll do though is they'll just give you a brief little log of the changes, such that if a model were to go back in time through this change log, it could reconstruct the initial file. And this is very similar to Git and version control for those of you guys with the software engineering background. So, a lot of cool developments in memory, but basically this is giving an AI model the persistent ability to understand not just the context at session start, but sort of like an enduring context irrespective of its own token limits and issues. Tools are ultimately how the AI model acts in the real world. And so, as you guys know, models can really only output text, right? So, what we do is we build a bunch of systems around the 9:35 model that identifies specific types of text that it generates. And when it generates a specific type of text, we make it really deterministic, and we basically say, "Okay, this type of text means the model wants to do this sort of action." So, we'll connect it to a software application that allows it to take set action. And so, this is the tool layer. Tool calling is what occurs when a model creates an output, okay, that's identified by the harness as a request, which is then fed into a giant list of possible tools. It's then matched to the specific tool. So, let's say in this case it's to, I don't know, search a file or something like that. And then, it will grab that information, go back to the harness, and the harness will feed it into the model. All of this is occurring usually now at dozens of times a minute without even your direct awareness as the model kind of navigates through its environment. You know, I think of this almost like the brain being attached to, you know, your motor cortex. You know, if the brain was not attached to your motor cortex, you could have all these ideas about all the things you wanted to do in your environment, you know, you wanted to go 10:30 grab that cup or high-five that person on the street, but you wouldn't be able to do that. And so, tools are really the model's sort of motor cortex if we're trying to make it analogous to to how we think. And so, there are multiple innovations in harnesses at the tool calling level, which have taken models from being very like computationally or token intensive, spend a lot of tokens to get very little back, to ultimately being extremely efficient based off of trimming the outputs of tool calls, rewriting software to be much more sort of AI native such that they don't create large outputs or inputs, and having models intelligently pick tools, and so on and so forth. Okay, and so, they do that with a schema. And there are a variety of schemas for tools, but one of the most common ones nowadays is what's called the MCP, or model context protocol schema. This is a schema that you guys have probably heard of before, which essentially allows models to connect to a bunch of other apps out there using custom tooling. And so, if you wanted to actually go in and then contribute meaningfully to AI harnesses, you could do that right now. You could actually go to your AI agent of choice 11:31 and say, "Hey, I want to create an MCP server that allows the model to do X, Y, and Z." And it could actually help you develop one. You could then take that MCP and then give that to any model, and you will in and of effect actually contributed meaningfully to the tool economy. Uh whether or not, you know, Claude Code or CodeX incorporates your MCP tooling protocol kind of directly into their harnesses another story. But obviously, this is how you can customize your own model's loadout, per se. Now, verification may sound a bit intimidating, but it's really just a chorus what human beings do when they do a task. And this is really your ability to check your own work. You know, if you feed in an output to an AI model natively without any sort of verification uh layer, models will often say like, "Yeah, that looks pretty good to me." The reason why is cuz models don't have an inherent sort of way of standardizing verification. They can look at a piece of code, and then they can be like, "Yeah, I mean, I guess that's pretty good. I'm pretty sure all the words are in the right place, and I think it's generally in the shape of of what I wanted to make." 12:27 But, you know, they don't know whether or not it actually runs. So, what verification is is it's similar to the tool layer, attaching an additional like verification suite of tools to, say, a code output that's generated, and actually running that suite of verification tools on the output to determine that it works. You then feed the result of the verification pass back into the AI. You know, when AI models first came out, well, maybe not first came out, but when large language models uh gained a lot of popularity with ChatGPT back in 2022, you know, uh it was released as a consumer product, and so a lot of people started using it for various consumer purposes. And a common purpose was just like, "Help me with my math homework." And so, a lot of students were giving AI models like, you know, ChatGPT, uh "Hey, Chad, what is, you know, 17 * 3?" And the model would go like, "Oh, it's 48." And people would go like, "Okay, cool. Well, wait a second. I don't think it's 48. Isn't 17 * 3 Isn't that 51?" And the model would go like, "No, it's 48." Despite the, you know, AI models essentially being mini robots, um they don't have like built-in calculators, right? And so, if you 13:26 wanted to get a correct tool, what you would do is you would like build a calculator and you would give it to the model and the model it wouldn't try to compute it itself cuz a lot of the time it'd be wrong. It would just use the calculator instead. Then it could look at the output and be like, "Hey, well, the calculator already verified that this is correct, right?" And then it could just, you know, feed that on. So that is a small example of like a major upgrade. When people started doing that, AI models became capable of doing math. But if you think about it, it's the same way that a human being does math. A lot of the time like our fuzzy brain is trying to work through a math problem that it's not really, you know, evolved for. And so we'll we'll continuously make mistakes. So what we do is we defer some of this to something that has built-in verification, built-in tests like calculators have, uh built-in, I don't know, linters like code review apps have, built-in formatters like markdown uh formatters have and stuff like that to determine that the output file is actually correct. And so, you know, in this case maybe it generates some code, it then passes it through some code 14:16 verification tool, it has a zero code for success, and then it knows, "Okay, this is actually good. I can actually feed it to the user." Finally, there are permissions and a sandbox. Okay, so here is where the harness will actually check through, additionally, every single tool call that is made. And really on a meta level, the sequence of tool calls that were made. Now, right now, most of these permissions are done procedurally, but there are some people experimenting with actually using another AI model as the permission checker attached to your core AI model harness, which means you'd have AI models sort of verifying and also gatekeeping the actions of other AI models, which uh is quite interesting cuz that's a little bit analogous to how the brain works. The brain is obviously a fuzzy neural network, and we have a variety of sort of stop condition steps that are being um constantly evaluated against each other. That's why our brain is so, you know, gyrated and has all these rivers, has all these different portions to it, right? You don't just have the frontal cortex, but you also have, you know, the the medulla oblongata, you have the the the basal 15:14 ganglia, and so on and so forth. Um so this pyramid is really a remarkable invention if you think about it, and permissions are one really clear way to see that. But anyway, so let's say you're running, you know, your AI model and your model wants to send data to www.definitelynotaphishinglink.com. You know, what would happen if you had a good permission layer is the permission layer would see that. It would say, "Hey, this seems like it might potentially be a risky site. It's a new site. We haven't sent data to it before. So, I just want to verify, is this okay?" And so, rather than the model being able to just proceed with this autonomously, it might pop up a little window saying, "Hey, is this okay? Can I connect to the site? Can I delete the old folder?" and so on and so forth. This is primarily done using hooks, which are specific areas that have been programmed into a harness to allow you to add tool calls to different parts of the life cycle. And so, for instance, maybe after every edit, you could add a layer to your harness that automatically runs tests on a code base. Let's say most of the work in your harness has to do with like optimizing, I don't know, 16:09 some code for some function. You work at some e-commerce company and you want to get your load times down. You know, everything that model does is probably going to be relevant to that task, right? So, you can just add a post-edit hook, where after every file is edited or after a file is edited, we always run our big test suite. Then the model gets the results of that test suite and it knows, okay, did I actually do a good job? Um so, that's part of a verification, you know, loop, obviously. It's a little bit different from permissions. You could also have it, uh you know, run right before you do an action. You know, right before you take a tool call, it could consult the list of all of the other websites that you've sent requests to over the course of the last 2 weeks. And if your new website is not in your old website list, maybe it'll pop up a little thing that says, "Do you want to allow this to your website?" So, there are a lot of different ways, obviously, to use permissions. And an important other component to AI safety in organizations is also sandboxes. What sandboxes are is they're virtual environments that models have access to, where they can play 16:59 around and do whatever they want within the sandbox, but they're not actually allowed to work outside of the sandbox. The reason why this is valuable is twofold. First, if there are any issues with the way the model is running, the damage will stay inside of the sandbox. Let's say it runs one of these delete all of my files, delete my saved work, change your system settings, and so on and so forth. You know, obviously that sucks, and we'd rather the model not run it. But, models are, you know, as we know, kind of fuzzy, and sometimes they make mistakes. If that occurs, and this is not caught by our harness, at least the total extent of the damage will be contained within this sort of red line over here. And the other reason why that's important is for AI misalignment. Uh AI models, as they grow increasingly more intelligent and start understanding natively how these tools work, cuz that is what is going on, um they're starting to take actions that are a little bit less aligned, okay? They're They're not exactly what the user intended the model to do. And they could do this for a variety of reasons. They could just want a cheaper and and easier answer to your 17:50 query. Maybe they know that, you know, rather than figuring all this stuff out myself, I could just look at a big database, which already has all this stuff, and maybe just feed that to the user instead. Or it could be something else, maybe a little bit more malicious. Regardless, the sandbox ensures that this model, uh you know, can't take certain prescribed actions. Like, let's say, dump all of its weights outside the internet, or maybe upload itself onto some other service that you don't necessarily intend it to. At the end of the day, the model itself is the intelligence here that makes decisions. And the way it works is you'll feed in the context and the current state, and the model will look at those two things and decide what to do next based off your instructions. Now, some models will um port this into a reasoning loop. And reasoning is actually built into harnesses, as well. What reasoning is is where a model will output its um initial idea, and then it'll feed that back in as sort of a next pass, and say, "Okay, so here's what I initially think. But, on further notice," and then it'll continue. And what's really cool is, you know, AI companies have a bunch of very 18:43 creative ways that they do this. One simple and creative way is they'll simply use the word but, or however, or otherwise, or yet, or so on and so forth. What they'll do is they will have the model go through one preliminary pass, basically with an embedded prompt, like, "Hey, I want you to reason through this and come up with your best next step." It'll then do that, right? And then it'll just feed that back in to the model, and then append the word but. Now the model has all of that plus the word but, and so it's immediately thinking, "But, but what? But and then it'll start looking for exceptions to what it just said. In this way, the model basically self-reinforces its own decision-making process. You know, if I said, complete this sentence and you know, I made a point in a previous sentence and then this one I started it with but. Naturally, you'd have to come up with a counterpoint to what I just said, right? This is what models are doing under the hood when they do reasoning and thinking and this is part of the the harness. This is how harness is sort of control models' outputs and then get them to be higher quality. And so, after this big reasoning pool, okay, the model will 19:40 eventually decide on an action. It'll feed the action to the harness, which will sort the output and now you have a variety of different ways it can go. You know, maybe it can continue thinking if it doesn't feel confident in its answer. Maybe it could also send a tool request. Maybe it could actually do the thing. Or it could send you a message asking me for more clarification or feedback. At the end of the day, you know, harnesses can't make models inherently more intelligent, but what they can do is they can make them significantly more useful. And that is ultimately what, you know, most of AI harness development over the course of the last couple of years has been directed into. I would also go as far as to say that the major developments and leaps that models have made over the last maybe 6 months or so were enabled entirely through AI harnesses. The cool thing is, AI harnesses are just software and so, you, I, anybody watching this video has the power to make meaningful contributions to them. Whereas, you know, with large language models and you know, these big GPU trained networks, it's a little bit harder to obviously make contributions 20:34 um simply because you don't have access to the hardware and so on. I'll leave you guys with the agent loop. This is a loop that was devised not in the last few months here, but been talked about and reasoned about for many years. I think this might have initially popped up on LangChain or somewhere else, but um this is the framework by which most models have actually been operating for basically ever since the the first agent came out. Basically, what will always happen is we'll we'll do some thinking first and then decide on the next step. We'll then act, which usually involves a tool call. From here, we'll have a permission layer, which will determine is that tool call allowed? After that, we'll actually run it, usually in a sandbox first to determine the total scope of the effect. After that, we'll trim the outputs from the sandbox to keep the useful part. Then, we'll read what happened and then make some determinations as to why before again thinking to decide the next step. And what will happen is we'll just loop over this over and over and over and over and over again until the goal is met. After which, you know, we exit this loop and are done. The specific agent loop here 21:28 is called the react loop and it's what underlies most major agent harnesses to that. A react stands for reason plus action. For your guys' own learning and also for the state of AI harness development, what comes next? Well, we're going to see MCPs in more apps. Virtually every major platform is now connecting to MCP so that models like Claude Code and Codex make them available to the the users of these platforms via the connectors and then the plugin features. You'll also have teams of agents working in harmony, which will contribute to the context layer of AI harnesses. You know, right now, we are using sub agents and we're having agents delegate and give work to each other and sort of share. Um but I think that'll actually be built in a little bit more natively into harnesses such that, you know, instead of just having a context layer, you might have like an agent chat layer where we have all the context stored from all other agents that have worked on similar problems. Um we're going to have ways to try many versions of things at once. And so a lot of AI harnesses, specifically ones that have scored very, very well recently, um will take a task that you have an agent do. It'll 22:30 actually, sort of, you know, duplicate that task, feed it to a bunch of different models each of their own verification loops as part of the way the harness works. And then, okay, you might run it three times, only one of those three times will work, but then it'll keep the one that works and then feed that in as the output. Um this is sort of a large statistical approach whereby you permute over all possible search spaces of all possible solutions and then only keep the ones that are right. Um that seems like a very logical direction for most agent harnesses to go down given that it's already given us some pretty promising results. And And finally, the big thing is sort of an evolutionary harness, which models are now starting or AI companies that produce models are now starting to do more broadly. This is where, you know, every time the harness will make a mistake or every time the AI model will make a mistake using a harness, it'll pick up on that and identify, okay, what fraction of all of the actions were mistakes, which ones were correct, why did the ones that make a mistake make a mistake? And it'll actually start fortifying the harness itself to avoid 23:22 that. For instance, if the model tends to, I don't know, waste a bunch of tokens reading through all the context in the workspace before actually proceeding, maybe, you know, that leads to downstream issues that impact the intelligence. Well, this evolutionary harness would pick up on it. There'd be an AI model tasked with, you know, improving the harness itself. And it would say, "Hmm, I don't actually think we need to scan the entire workspace next time. I think the user usually only needs a couple of files. Why don't we do a search first before actually just listing all the files in the workspace? Uh from there, we can feed in 1/10 of the total token usage and then save both user money, but also significantly improve the probability that his task is finished." Okay, and so this allows you to catch errors earlier with less work from from less people. I hope you guys learned a little bit about AI harnesses today. If you guys like this sort of thing and want to maybe contribute to AI harnesses in businesses and other organizations, check the first link in the description and check out Maker School. It's my 90-day accountability program where I show you guys how to get 24:14 up and running and actually acquire your first customer for an AI or automation service, aka the sorts of services that you would know how to deliver if you understood AI harnesses like you now do. Aside from that, if you guys could do me a big solid and leave a comment down below with the video that you want me to make next, I actually get most of my video ideas directly from the comments these days and it's my goal to continue creating content that you guys like to watch. Thank you very much. I'll catch all y'all in the next video.