Collect Everything, Trust Almost Nothing

Over the past few weeks I taught my market tooling to listen to roughly ten different crowds at once. The bull-and-bear chatter on a trading message board. Raw forum posts pulled from public archives, which I score for sentiment myself instead of trusting anyone's black-box count. Front-page tech chatter. Search trends. Encyclopedia page views, which turn out to be a wonderfully unglamorous attention signal. Insider filings. Video-platform chatter. A news-tone feed. Every lane lands in one store, on one schema, stamped with where it came from and when.

The obvious next move, the one every side-project instinct screams for, is to feed all of that into the engine that grades stocks every night. More signal, better grades. That is the move I refused, and the refusal is the only part of this build worth writing about.

A one-way wall#

I wrote before about the boundary inside my finance engine: the data trusted enough to composite into a grade is walled off from the data trusted only enough to watch, and a test keeps the crowd from ever moving the number. This project extends that same line around the entire collection layer. Everything the tracker gathers may flow into analysis, dashboards, and questions. Nothing flows backward into the grade. The wall runs one way, and the grading engine has no path through it.

The reasons are the same ones from the original post, just louder at this scale. Every one of these feeds is unaudited. Some are gameable by a motivated stranger with a few accounts. The counting services are black boxes that change their methodology without telling you. The endpoints themselves churn, rate-limit, and disappear, and a signal that vanishes mid-month is not a signal you can hang a grade on. A grade has to be explainable after the fact from inputs I can defend. "The crowd got loud" is not a defensible input. It is exactly the noise the grade exists to be calm against.

Making the wall structural#

The rule would be worthless if it lived in my memory, because convenience erodes memory. So the wall is built into the shape of the system, the same trick that made a compliance rule impossible to break in my visualization work: the safe path is the only path that exists.

The tracker is a separate codebase with its own store. Its fused score is its own number, computed by its own pipeline, published to its own file. There is no import path, no shared table, no code route by which that number reaches the grading engine. One source needed special care: the trading message board already feeds a small display widget elsewhere in my system, so rather than extend that existing pipeline and risk entangling it, the tracker got its own clean-room client. Two clients for one API is mildly wasteful and completely safe, and I will take that trade every time.

Inside its own walls the tracker is allowed to be opinionated. Each source's scores are standardized and clipped so one hysterical feed cannot dominate. The fusion has a floor on the number of independent sources that must agree before a composite exists at all; below the floor, the day simply has no composite, because a number backed by one noisy feed is worse than no number. Absence beats fabricated confidence. That principle has survived every project I have applied it to.

What collecting is actually for#

If none of this may touch the grade, why collect it at all? Because watching is genuinely valuable, it is just a different job than grading. Divergence is the interesting product: when the crowd gets loud on a name and the insider filings stay quiet, or the reverse, that is a question worth investigating, and I built a small public toy around exactly that tension. Attention data also answers questions grades never could, like whether a name is being discovered or abandoned, and whether a move arrived with chatter or in silence.

And there is a legitimate gate through the wall, with a lock on it. If the tracker's composite ever earns a place in grading, it gets there the way any signal must: pre-registered, tested forward on out-of-sample data, judged by the same rules that keep a backtest honest, and admitted as a deliberate, versioned decision. The wall does not say never. It says not through convenience, and not silently.

The portable version#

Strip the finance from this and one design question remains, useful anywhere data flows near a consequential number: for every feed you add, write down which numbers it is allowed to touch, and make the default none. Collection is cheap and greedy by nature; consequence should be expensive and deliberate. The systems I trust most are the ones where those two appetites are physically separated, where you can add a tenth source on a whim precisely because there is no path by which a whim reaches the output that matters.

Collect everything, and let all of it reach the watching side. Keep the grade behind the wall, with one gate, and open it only on purpose.

---

Related:

• Composite What You Trust, Watch What You Don't, the original boundary this post extends outward
• Insiders vs Crowd, the public toy built on the divergence question
• How to backtest without fooling yourself, the gate any signal must pass to earn consequence