In the time I've been in the trading space, I've seen many trends come and go. Most strategies are marketed as discretionary, which makes them almost impossible to disprove, and the ones marketed as "mechanical" usually stop being mechanical as soon as they start losing.
Something that's become more and more prominent as of late however is this idea of statistics, i.e. knowing the odds of a certain event occurring. You might've heard a few of these: IB (initial balance) and the odds of one or both sides breaking and in which order, "gap fills" and the odds of them being mitigated, and more. Now there isn't inherently a problem with these ideas, but the way they're presented makes it seem like a high percentage must contain an edge, while one important comparison is left out.
That missing comparison is the subject of this essay. We ran these measurements on fourteen years of one-minute index data, but the idea itself is simple, and you don't need a background in statistics to understand it.
What is signal (and what isn't)
§ 1Suppose a service tells you that something happens 62% of the time. That may be useful, but the percentage alone can't tell you, because you first need to know how often the same result would appear by accident.
Imagine 1,024 people flipping a fair coin twenty times each, where nobody has an edge. Even so, someone in the room will usually land fifteen heads or more, and that person can show you a real 75% hit rate. The count is accurate, it just doesn't mean what it appears to mean.
The same selection happens in trading. Thousands of people test different levels, time windows, filters, and setups, the results that look good get posted, and the others disappear. By the time a statistic reaches your feed, it has usually been selected because it looked impressive.
You see the winner. You rarely see how many ideas were tested first.
A statistics service can test far more combinations than any one trader, since it can split the market by level, time window, weekday, direction, and dozens of other conditions. With 432,000 combinations to choose from, random data is expected to produce a best result near 80%, which means an 80% cell can appear even when none of the tested ideas works.
Comparing stats to their baselines
§ 2Gap fills. In our data, gaps fill about 62% of the time, which at first glance looks like evidence that price is drawn back to the gap. Then we measured the same distance in the opposite direction, where there was no gap, and price reached that level 59% of the time. Most of the 62% came from normal intraday movement, and the gap itself added roughly three percentage points.
IB breaks. The initial balance is the first hour's range. The odds that one side eventually breaks are high, but that is partly built into the question, given that for neither side to break, price has to remain inside the first hour's range for most of the session. The break rate describes how markets move, it doesn't automatically give you a trade, and you still need to check whether the order, timing, or follow-through does better than you would expect without the pattern.
Every year green. A result that worked every year feels much harder to dismiss, so we tested that idea as well. When we searched data built to contain no edge, about 95% of the searches still produced a best result that was green every year, which means that if you try enough variations and keep only the winner, consistency can appear by accident too.
This is why the number of tests matters. A result from one rule chosen in advance is very different from the best result found after trying thousands of rules, and as the search gets wider, the best result rises even when every rule is random:
We ran a search of roughly 34,000 rule combinations over the fourteen-year sample, and the winner looked strong and was profitable in every calendar year. We then repeated the same search on data where the supposed edge had been removed, and a result that good appeared about one time in four. The original result looked rare until we compared it with the full search that produced it.
The missing piece of the puzzle
§ 3The missing comparison is called a baseline, and it answers a simple question: if the pattern had no effect, what result would we normally expect?
To build one, we keep the same market sessions but shuffle the part the claim says matters, so the volatility, ranges, and number of opportunities stay the same, and then we run the count again. Repeating that process a few thousand times shows us the range of results that can appear without an edge.
If the observed result sits near the middle of the shuffled range, the pattern hasn't shown anything unusual, and if it lands far beyond nearly all of the shuffled results, we have evidence that the pattern contains information. The headline percentage only becomes useful once we know where it sits against that comparison.
A larger sample doesn't solve this on its own. It gives you a more precise count, but an untested count can still measure something that happens naturally, and a baseline is what separates "this happened often" from "this happened more often than we would expect."
Conclusion
§ 4I still think statistics are the most honest way to study trading ideas, and measuring the odds isn't the problem. The problem is stopping at the raw percentage and calling it an edge.
At Q1Tile, we compare each statistic with shuffled versions of the same sessions, and we also account for how many rules were tested and whether the result survives realistic costs. Many promising patterns fail those checks, which is exactly why the checks matter.
When a statistic catches your attention, ask how many ideas were tested, what the result was compared against, and whether the rule was chosen before or after the data was examined. If those questions can't be answered, the headline percentage isn't enough.
