We may earn a commission through these partner links. Demo funds are virtual; access and country eligibility depend on the platform terms. Disclosure
A rule evaluated on the same observations used to design it can look stronger than it is. Out-of-sample testing preserves data that did not influence the final rule.
Define what counts as design
Choosing indicators, thresholds, timeframes, symbols, sessions or exclusion rules after inspecting results is part of development. Even a manual adjustment based on a few losing charts uses information from those outcomes.
Record those choices. Calling a rule ‘simple’ does not make its selection process independent.
Use a chronological split where appropriate
An illustrative dataset might allocate an earlier sixty percent to development, the next twenty percent to validation and the last twenty percent to a final test. Those percentages are examples, not universal optimal settings.
For a time-dependent process, a random shuffle can mix information from neighbouring observations. Consider how entries overlap and whether their outcomes extend across a split boundary.
Keep a clear role for each segment
| Segment | Permitted purpose |
|---|---|
| Development | Explore and define the rule |
| Validation | Compare documented candidate choices |
| Final test | Evaluate the frozen selected rule |
If the final result is used to change the rule, it has become development information. A new independent evaluation is then needed.
Freeze the whole economic model
Specify accepted entry timing, expiry, payout assumptions, costs, tie handling and missing-data treatment. A rule is not fully frozen if its chart condition is fixed while favourable payout values are chosen later.
Use actual historical payouts when available. If they are missing, show scenario results and label the assumption rather than reporting a verified historical profit.
Track all tested alternatives
Testing fifty variants and publishing only the best makes the result difficult to interpret. Preserve the number of candidates and their selection criteria.
A final evaluation should include all qualifying observations, including inconvenient sessions, unless an exclusion was defined independently of their results.
Interpret success and failure proportionately
A positive test is evidence about one sample under stated conditions. It does not guarantee stability, live execution quality or payment reliability. A negative test is useful information rather than something to hide by redesigning the report.
TradingView’s strategy documentation discusses backtesting limitations and biases. Platform output still needs a clearly specified research process; a performance chart does not make that process automatic.
A fictional split with a clear decision boundary
Imagine 1,000 chronologically ordered decision events. Events 1–600 are used for development, 601–800 for comparing predefined candidates, and 801–1,000 for the final frozen evaluation. This is an illustrative design, not a universal sample-size recommendation.
Before event 801, retain the selected settings and the complete economic assumptions. If the final segment’s losing examples lead to a new filter, that segment has influenced design. Label the revised rule as a new version rather than describing the same final segment as untouched evidence.
Record a freeze manifest
| Component | What to freeze |
|---|---|
| Data | Instrument, source, dates, timezone and exclusions |
| Calculation | Indicator inputs, warm-up and bar state |
| Decision | Exact entry condition and observable signal time |
| Economics | Stake basis, payoff, costs and tie handling |
| Evaluation | Metrics, sample boundary and stopping rule |
| Version | Saved rule text or code identifier and date |
A frozen indicator threshold is insufficient if the analyst later removes losing sessions or changes the assumed entry price. Treat every outcome-dependent decision as part of the selection process.
Lookback data and outcome leakage are different
An indicator at the beginning of a test may legitimately need earlier price history to initialise. That is different from using later outcomes to tune its settings. Preserve the history available before each decision and keep its role separate from performance labels.
Boundary-crossing trades need explicit treatment. Suppose a decision occurs near the development period’s end but its expiry falls inside validation. If the full outcome informs development, information from the later period has entered the earlier selection. Document how such overlapping observations are handled rather than assuming a date split solves everything.
Walk-forward testing requires a prior procedure
A walk-forward design can repeatedly develop on an earlier window and evaluate on the next window, then advance through time. State the window lengths, update schedule and selection method before comparing results. Keep each evaluation window separate from the decisions used to build its rule.
This is not permission to re-optimise until every window looks attractive. If the window design itself is selected after seeing all results, disclose that additional selection. Aggregated performance should include all scheduled evaluation windows, including weak ones.
Separate historical simulation from forward observation
TradingView’s strategy documentation describes simulated historical and realtime testing and discusses bias and overfitting. A historical test can be out of sample relative to development without being a live execution test. A forward demo observation can also differ from funded execution.
State which evidence you actually obtained. Neither category establishes withdrawal reliability or local product permission. A realistic research design keeps those operational questions in a separate evidence checklist.
A negative result is a useful result
If the frozen rule fails in its evaluation segment, retain the result. A later revised version may be worth studying, but it does not erase the earlier finding. Versioned records prevent an endless succession of “final” tests from becoming invisible tuning.
Show complete counts, net results and assumption limits. A favourable win rate alone cannot establish profit without the payoff model. If historical payouts are unavailable, use labelled scenarios rather than claiming measured returns.
Frequently asked questions
Does a later date automatically mean out of sample?
No. If its results influenced any part of the rule or selection, it is not independent final evaluation for that version.
Can I tune on the final test and still call it final?
Disclose that it became development information and seek new evaluation observations. Renaming the same data does not restore independence.
Is a random split always wrong?
No universal rule is given here. For time-dependent trading data, examine chronology, overlapping outcomes and the information available at decision time before choosing a split.
Does a positive test guarantee future performance?
No. Report its scope and limits. Our research methods hub and methodology distinguish a sample result from a guarantee.
Sources and research scope
Public-source review: 25 October 2026. Examples are illustrative. Documentary research does not represent funded-account testing. Rules must be checked against the current product, entity and customer location.
Related research
Research methods · Research methodology · Calculation tools
Risk: Fixed-outcome contracts can lose the entire stake. No chart pattern, calculator or demo result guarantees future returns.