The Answers I Didn't Trust

Last year I asked a chatbot what my VO2 max should be at 40. It gave me a clean number and a confident paragraph around it, and I noticed I was about to just believe it. So I did the thing I should have done first and asked where the number came from. It came from nowhere I could point to. Somewhere in the average of everything ever written about VO2 max, weighted toward whatever the model had seen the most of.

For most questions that’s fine. For questions about my own body, where being wrong doesn’t cost me a bad paragraph but a few years of healthspan, it isn’t. That gap is the whole reason CortexPanel exists.

The average of the internet has a confidence problem

A general model answers a longevity question in the same confident voice it uses to pick a pizza topping. It sounds sure, and it shows you nothing you can check. The trouble is that on health the loudest content is usually the least rigorous. It’s the SEO winner, the supplement affiliate, the person optimizing for reach instead of for being right. When you flatten all of that into one answer, the confident tone survives and the rigor doesn’t.

I’ve written before about how an AI interface quietly decides whether its output reads as a suggestion or a command, and how most of them default to command. This is the same problem with higher stakes. I didn’t want a command. I wanted to know who was actually talking.

What it is

CortexPanel is a panel of experts you can ask real questions of. You pick a domain, health is the most built-out so far, with finance and psychology partway there, and instead of reaching for the open web it answers from a corpus I curated by hand. Peter Attia, Andrew Huberman, Tim Ferriss, Rhonda Patrick, and a bench of medical specialists behind them. Roughly 65,000 chunks of their actual work, embedded and searchable, with every answer citing back to the source it came from. When I ask what my VO2 max should be at 40, the answer comes back framed the way Attia frames it, with a link to the episode or the page in Outlive where he says it.

One thing I learned building the retrieval: raw vector similarity is a liar. A nutritionist with one stray sentence about cholesterol would outrank the actual lipidology expert whose relevant thinking was spread thin across thousands of chunks. I had to build a blended ranking that weighs how central a topic is to a given expert, not just how well a single passage matches the words in your question. Getting the right person to answer turned out to be most of the work.

The product is the curation

Here’s the part I’d defend hardest. The value isn’t the model. Any decent model plus a vector database gets you a working demo in a night, and mine did, at 2 AM. The value is the list of who’s on the panel and who isn’t.

Almost every “ask an AI expert” tool works by scraping the web for expertise and hoping volume averages out to quality. CortexPanel does the opposite. It curates expertise into the system on purpose, filtered for integrity and signal density rather than popularity. That filter is the hard part, and it’s a human judgment I can’t automate away. Some genuinely popular voices don’t make the cut because they sell more certainty than they’ve earned. Deciding to leave them off, and being able to say why, is the actual product.

No affiliate links, on purpose

There’s a layer I’m still building called Picks: specific supplements, tools, and products for a given situation. It will never be tied to affiliate revenue or sponsorship. That decision costs me the obvious way this kind of thing makes money, and I’m keeping it anyway.

The reason is simple. The moment a recommendation earns me a cut, curation quietly turns into advertising, and I built the entire thing to get away from exactly that. A panel I trust with questions about my own body can’t also be a storefront. Once you know the recommendation pays the recommender, you can’t unknow it, and neither can anyone else.

Where it’s wrong

I don’t want to oversell it, so here’s the failure rate. I built a formal accuracy audit: 56 questions across five experts, scored by a separate model on grounding, retrieval quality, confidence calibration, and citation correctness. The baseline pass rate is 79%. Attia and Lex Fridman score 100%. Rhonda Patrick scores 30%, and the reason is boring and honest: she only has about 1,200 chunks of content in the system against Attia’s 17,000, so the panel simply doesn’t have enough of her to retrieve. The most common failure across everyone is citation correctness. The answer is good and the citation is in the right neighborhood, but it doesn’t always land on the exact claim.

Why publish the number? Because a health tool that hides its error rate is worse than no tool. If I’m going to ask people to trust curated answers over the confident average, the least I can do is publish where the curation runs out.

Why I keep working on it

It isn’t launched. It has one user, which is me, and the VO2 max question is one I genuinely asked it. I built it because the honest answers to the questions I actually care about were scattered across a dozen people’s life’s work, and none of the tools promising to synthesize them were built to keep the sources honest. So I built the one I wanted to use. That’s the whole thing. If it ever ships, it ships on the same terms it runs on now.