Quite often when I encounter a newly built responsive site, one of the first things I do is check out the size (in KB or MB) of the homepage. More often than not I find that I discredit the site because the page is 2MB or more. I often wondered how can you proclaim your new ‘responsive build’, but neglect something so fundamental as the size of the pages you serve.
My opinion was based on much of the advice I had encountered online, but more than that I thought it was based on common sense. The reason we make make websites responsive is to make them more easily available to mobile devices. Mobile devices are reasonably likely to be using a mobile connection which is going to be slow, so part of making a site accessible is being frugal with page sizes.
I decided to experiment with a handful of responsive sites by competitors (and our own!) to gain more of an insight into what effect page size has. Very rarely do I get out the office to test our sites, but I took myself and my mobile out, along with a list of sites to see how they would be delivered on a mobile connection.
Measuring page size on a mobile
To perform an authentic test I wanted to measure page size on the mobile itself. It wasn’t something I had done previously, so a little experimentation was required to determine the best way of measuring directly on my mobile.
My initial thought was that I could use the O2 account management app I have installed on my mobile. It provides a ‘metre reading’ of data usage for the current billing period. I quickly realised that the report it generates is not real time, and that having to wait 30 minutes or more to see an update was unrealistic. Fortunately there are a couple of apps on the App Store which can provide real time data usage reports. I used one named DataMan which worked a treat for me.
In order to get a fair reading I restricted my mobile data to be available for my browser only. It’s important to do so, as other apps might be operating in the background. I then cleared all browser history and cache, and reset the counter on my data usage app. In theory only the data from my browser should be recorded in the usage tracker app.
Turning my back on 4G
I have a new iPhone, and luckily for me Edinburgh has a reasonable 4G network. It’s lovely and fast, but I didn’t want to perform my experiments using the best mobile connection I have available. What I really wanted to do was test on a connection that is similar to what you would experience outside a major city. In order to do so, I disabled my 4G connection, and walked to the top of Arthur’s Seat, Edinburgh’s little mountain. It did the trick, I was using 3G, but the connection was flaking out on me, and actually the phone was using an Edge connection.
It’s pretty chilly on the top of Arthur’s Seat at this time of year, so I quickly set to work testing the list of sites I had prepared. I tested each site, setting a one minute time limit for the homepage to load.
Firstly I was pleased to discover that for the most part, page sizes were noticeably smaller when viewed on a mobile versus what they were when viewed on a desktop. It was fairly common to find that the page size was 1Mb smaller than the desktop version. Sites that were 2-3MB on the desktop dropped down to 1-2MB, and sites that were 1-2MB generally dropped down to around 800KB.
A web page that is under 500KB is at a greater advantage than one that is over that amount. I found during my experiment that at below 500KB I could quite easily view the entire homepage, and likely view other pages if desired. My connection during the test was producing about 500KB per minute, which would explain why pages below that size were downloaded in full.
Web pages between 500KB and 2MB behaved very similarly. Given my connection the entire page was not going to load, but what I found was that the markup and styles would at least load. As long as the important content was text, then I could access the information I needed.
For some reason, any pages above 2MB would just not load at all. I don’t know what caused this, and I would have thought that the styles and markup would have been loaded just as they had done on pages in the 500KB to 2MB range.
Sites that use non local fonts (i.e. use web fonts from services such as Typekit) delay the display of text on the page. This was my biggest complaint of our own sites. In our case, the solution is to make use of Font Events as part of Typekit’s service. Web fonts are only utilised when they are fully loaded. Device fonts are used while the web font is loading.
What did I get from this experiment?
When I started this experiment, the reality was that I wanted to validate the ‘default’ aversion I have to bloated responsive websites. I’m not sure I succeeded in that respect. I wanted to see 1MB+ web pages fail to load on mobile connections. Instead I discovered that a lot of the sites which fall into this category coped reasonably well.
At this point I should mention that my test slightly favoured so-called functional websites. That is, sites that a user would visit to help them fulfil a task (for example, making a reservation at a hotel). Such sites need to be accessible to perform their primary functions. Do all responsive website fall into this vein though? No they don’t, and it’s not fair to penalise all responsive websites if they happen to be larger than 500KB. I’ve realised that I was being far too single minded, and I was failing to take into account what individual sites hope to achieve. For example, one of the first responsive sites Primate developed was Harviestoun, which isn’t highly optimised, but it is a very successful website.
I was really pleased to see that page sizes dropped on mobiles, but equally disheartened by the excessive assets added to desktop versions. Does that mean that developers understand the importance of reducing a mobile sites footprint, but then neglect the desktop version? I think that would be fair assessment to make.
Personally I’ve really enjoyed a more practical testing of responsive sites. It’s whet my appetite for getting more realistic about results, instead of operating on assumptions.