First and foremost, we believe that speed is more than a feature. Speed is the most important feature. If your application is slow, people won’t use it...
We think that the application has to be fast, and if it’s not, you can see what happens. We have every single one of our portfolio company services on Pingdom, and we take a look at that every week. When we see some of our portfolio company’s applications getting bogged down, we also note that they don’t grow as quickly. There is real empirical evidence that substantiates the fact that speed is more than a feature. It’s a requirement.
In the spring of 2010, tech VC Fred Wilson said this about the web applications that his company, Union Square Ventures, supports. Considering the fact that Union Square backs household-name winners like Etsy, Meetup, and SoundCloud, it's safe to assume Fred knows what he's talking about.
Three years ago, Joshua Bixby wrote this great post that validated Fred's statement. Joshua randomly selected five sites backed by Union Square and ran them through WebPagetest, and found that the median site took 4.5 seconds to fully load. Pretty good, especially compared to the then-benchmark of around 7 seconds for Fortune 500 sites.
Recently, someone asked me if VC-backed sites still outperform other sites, and I thought it was high time to revisit this question.
Methodology
1. Per Joshua's study, randomly select five sites currently backed by Union Square. (I've anonymized these, so as to avoid embarrassing anyone.)
2. Using WebPagetest, test each site's home page nine times - first as the page would load over DSL and then over cable - via the test server in Dulles, VA. (While cable has overtaken DSL in usage, I feel like it's too early to disregard DSL completely. And as we'll find out in a moment, the DSL results are pretty compelling.)
3. Take the median load time for each page and plot it next to Joshua's 2010 test results.
4. New in 2013: Using the frame-by-frame filmstrip view of the median load time for each page, identify the median time to interact. (TTI is the point at which the primary page content appears and becomes interactive, e.g. a feature banner that contains a call-to-action button.)
Results
This is where I'll let you in on my under-the-vest hypothesis: based on the recent spate of findings that pages are getting bigger and slower, I somewhat-more-than-half-expected to learn that pages had slowed down somewhat, even in this set of sites. But the results were more striking than I'd expected:
- The current median load times over both DSL (11.44s) and cable (5.15s) are slower than the 2010 load time (4.54).
- The median time to interact over DSL (5.6s) is slower than the 2010 load time (4.54s).
- While the median TTI over cable is relatively fast at 3.5 seconds, it's not optimal. Ideally, pages should be interactive in less than 3 seconds.
- The median page was 1520KB and contained 100 resource requests. Unfortunately, we don't have these numbers from 2010, but it's a fairly safe assumption that this is an increase over three years ago, and that this parallels the growth we've seen elsewhere in the Alexa Retail 500 and in the sites tracked by the HTTP Archive.
| | Load time - DSL (2010) | Load time - DSL (2013) | Load time - Cable (2013) | TTI - DSL (2013) | TTI - Cable (2013) |
| Site 1 | 3.30 | 4.37 | 2.45 | 4.3 | 1.3 |
| Site 2 | 3.98 | 7.59 | 4.50 | 5.3 | 3.1 |
| Site 3 | 4.54 | 11.44 | 5.15 | 5.6 | 3.5 |
| Site 4 | 4.91 | 12.33 | 5.32 | 8.2 | 4.6 |
| Site 5 | 5.87 | 16.82 | 16.74 | 10.1 | 4.6 |
| Median | 4.54 | 11.44 | 5.15 | 5.6 | 3.5 |
If tables aren't your thing, check out the graph versions, first showing the difference between the 2010 load time and the 2013 load times for both DSL and cable:
You can see that, across the board, load times over DSL are dramatically slower. Load times over cable vary between being somewhat faster and significantly slower.
And now, for interest's sake, let's look at how the current TTIs compare to the 2010 load time:
It's interesting to note that, while in theory TTI should be significantly faster than load time, in these tests the TTI over DSL was still markedly slower, while the TTI over cable was only marginally faster in most cases.
Caveat and conclusions
This is obviously a pretty simple ad-hoc set of tests. We're looking at just the home pages, so we can't measure flow or see how pages perform deeper into the visit. And we're only looking at five sites. But it's still telling to look at these simple numbers side-by-side with the numbers from five years ago.
I'm drawing attention to this, not to shame site owners or VCs, but to highlight the persistent performance challenges presented by page growth and complexity. Even hot household-name sites aren't immune to these challenges.