A useful place to start with backtest overfitting and the Sharpe ratio is this: the ratio is an estimate, not an observed fact. According to the CFA Institute research abstract on the statistics of Sharpe ratios, its building blocks (expected returns and volatilities) are unknown quantities that must be estimated statistically. The same abstract says that monthly Sharpe ratios cannot be annualized by multiplying by √12 except under very special circumstances. It also says that in an illustrative example, the annual Sharpe ratio for a hedge fund can be overstated by as much as 65 percent because of serial correlation in monthly returns.
The sources reviewed for this explainer do not measure backtest overfitting directly. They cover two related topics: how Sharpe ratios are estimated, and how US Securities and Exchange Commission (SEC) materials treat performance in investment adviser marketing. Where this article goes beyond what the sources say, it is labelled as our interpretation or framed as a question for the reader.
Why a backtest Sharpe ratio is an estimate
The CFA Institute research abstract states that the inputs to a Sharpe ratio, expected returns and volatilities, are unknown quantities that must be estimated statistically. We read only the abstract, and this article claims nothing beyond it.
Our interpretation, not a finding of the paper: if the inputs to a Sharpe ratio are estimates, then the Sharpe ratio printed at the end of a backtest is also an estimate. It should not be read as a fixed property of a strategy. These points are raised here only as context for the reader.
A reader who sees a single Sharpe ratio from a backtest could ask:
- Which data and which period were used to estimate the expected return and the volatility?
- Is the figure shown with any indication that it is an estimate, or as a single number with no context?
These questions do not test whether a backtest is overfitted. They only keep the statistical nature of the ratio in view.
Annualizing a Sharpe ratio: the √12 shortcut
The CFA Institute research abstract states that the author shows monthly Sharpe ratios cannot be annualized by multiplying by √12 except under very special circumstances.
The abstract read for this article does not let us describe those circumstances in detail, so we do not try to. What follows from the claim is narrow: multiplying a monthly Sharpe ratio by √12 is not a conversion that holds in general.
Clearly labelled conceptual example (synthetic, no real data): imagine a backtest report that shows only an annual Sharpe ratio and notes that it was obtained by multiplying the monthly value by √12. Based on the abstract, a careful reader would treat that annual figure as depending on an assumption that holds only under very special circumstances. The example contains no numbers on purpose. It illustrates a question to ask, not a measured result.
Serial correlation and an overstated Sharpe ratio
The third point in the CFA Institute research abstract is about serial correlation in monthly returns. In an illustrative example, the annual Sharpe ratio for a hedge fund can be overstated by as much as 65 percent because of serial correlation in monthly returns.
It is worth keeping the exact strength of that statement:
| What the abstract says | What it does not say |
|---|---|
| The example is illustrative | That every fund or backtest shows the same effect |
| The overstatement can be as much as 65 percent | That the overstatement is 65 percent in each case |
| The cause named is serial correlation in monthly returns | Anything about backtest overfitting as a cause |
Our interpretation: the illustrative example concerns a hedge fund, not a backtest. Readers may still find it relevant to backtests that report annualized Sharpe ratios from monthly returns, but the abstract does not make that link. Asking whether the monthly returns behind a reported figure show serial correlation is a reasonable question. It is not proof that any particular figure is overstated.
Sharpe ratio and the SEC investment adviser marketing rule
SEC staff recognize that advisers may be unsure whether metrics such as the Sharpe ratio are performance under the rule. This is why the regulatory context is relevant. The SEC small business compliance guide on investment adviser marketing gives the following background:
- On December 22, 2020, the SEC adopted amendments under the Investment Advisers Act of 1940.
- The amendments reflect developments since the advertising rule's adoption in 1961 and the cash solicitation rule's adoption in 1979.
- The marketing rule contains seven general prohibitions that apply to all advertisements.
- No written agreement is required when the promoter receives de minimis compensation of $1,000 or less, or equivalent non-cash value, during the preceding twelve months.
- The SEC set a compliance date of November 4, 2022, eighteen months after the effective date.
The key dates, from that guide and from the SEC staff marketing compliance FAQ, are:
| Date | Event |
|---|---|
| December 22, 2020 | The SEC adopted the amendments |
| May 4, 2021 | Effective date; an adviser may choose to comply with the amended marketing rule in its entirety any time from this date |
| November 4, 2022 | Compliance date |
What SEC staff say about the Sharpe ratio as performance
The SEC staff marketing compliance FAQ speaks to the Sharpe ratio directly. The staff recognizes that advisers may be unsure whether characteristics such as volatility, the Sharpe ratio and the Sortino ratio are performance under the rule. This article does not say how that question is resolved, because the supported claims do not cover the answer.
The same FAQ also states:
- The FAQ responses represent the views of the staff of the Division of Investment Management and are not a rule, regulation, or statement of the SEC.
- The rule requires gross performance to be accompanied by net performance calculated over the same time period and using the same type of return and methodology.
- The staff believes a reasonable period to calculate performance results based on the most recent calendar year-end generally would not exceed one month.
- Advisers are required to maintain a copy of all compliance policies and procedures in effect at any time within the previous five years.
Our interpretation: the staff's own statement that advisers may be unsure about metrics like the Sharpe ratio is a reminder that the regulatory status of such figures in marketing is not something a reader should assume. Anyone with a compliance question should consult the primary documents and a qualified professional, not this explainer.
Questions to ask when reading a backtest Sharpe ratio
The list below brings the points above together as questions. It is not a method for detecting backtest overfitting, and answering these questions does not show that a strategy works.
- Is the Sharpe ratio presented as an estimate, given that its building blocks must be estimated statistically, according to the CFA Institute research abstract?
- If the figure is annualized from monthly returns, was it obtained by multiplying by √12, a conversion the same abstract says does not hold except under very special circumstances?
- Could serial correlation in monthly returns be relevant, given the illustrative example in that abstract?
- In adviser marketing, is gross performance accompanied by net performance over the same time period and using the same type of return and methodology, as the SEC staff marketing compliance FAQ describes the rule as requiring?
Limits of this article
This article is educational information only. It is not investment advice and does not recommend any strategy, product or asset. It does not measure backtest overfitting, test any method or report results of our own. It does not cover the full Sharpe ratio paper, which was read only as an abstract, or give legal or compliance guidance.