While we’re still recovering from full bellies and empty wallets from the Thanksgiving celebratory weekend, we poured over the performance results for each site to drill in to see how they compared to last year’s event.

As usual, we used AlertBot’s external website monitoring system and its TrueBrowser™ technology to monitor all three sites from Thanksgiving Day through Black Friday and Cyber Monday, spanning from November 22, 2018 to November 26, 2018. We expected strong, reliable performance again during the entire run and we were not disappointed. The results were nothing short of impressive. In fact, we were impressed to mostly see improvement this year over last year.

Reliability

For the reliability evaluation of a website, we look for failure events (like when a page doesn’t fully load or it is completely down), and we look for what caused those failures.

Last year, in an unusual feat, each site experienced not a single error or failure event. The same mostly held true for 2018, but both Walmart.com and Target.com struggled with a few slow file load times (which can cause a page to load slower), but it was never enough to cause any actual site downtime. With that in mind, we think it’s still fine to award 10’s across the board.

(Amazon 10/10)
(Walmart 10/10)
(Target 10/10)

Speed

When evaluating a website’s speed, we look at the time it takes the site’s homepage to render and load to the point of being fully interactive by the user. These tests are performed from the perspective of a first-time visitor with no prior cache of the website’s content. AlertBot runs the tests inside real Firefox web browsers using AlertBot’s TrueBrowser ™ monitoring.

Last year was the first time we ran this event, so it was interesting to be able to compare last year’s results with this year’s. Ecommerce sites tend to have very graphics-heavy designs, and especially with sale events like these, the graphics are often big, frequently changing, and sometimes even animated or video-driven. (Amazon even had live video streaming at one point throughout the purchasing frenzy!)

With that said, through Amazon.com’s 5-day run, they saw the fastest day, on average, to be Sunday, November 25^th with 4.2 seconds—which is almost exactly what last year was (Their fastest was also a Sunday at 4.3 seconds). Their slowest day, on average, was actually on Black Friday itself at 4.5 seconds, which, admittedly, still isn’t too bad. When looking at specific times of day for performance, the best hour was 7AM on Sunday with an impressive 2.6 seconds (an improvement over last year by almost a full second), while the day before saw the slowest hour at noon with a dismal 9.3 seconds (which was significantly worse than last year).
(Amazon 9/10)

Walmart.com was the fastest last year and proved not only to hold that title again this year, but they also showed improvement! Their best average day was Cyber Monday, November 26th at 3.8 seconds. Their worst day on average was Sunday, November 25^th,  at 4.1 seconds (Coincidentally, it was also Nov. 25^th last year, but this year it was almost a full second faster). Finally, their best hour on average was on Cyber Monday at an impressive 1.8 seconds at 6PM. Their worst time on average was 6.9 seconds at 5PM on Black Friday, which is not when you want to be experiencing your slowest web speed.
(Walmart 9.5/10)

Last, but certainly not least, Target.com performed respectably, but once again underperformed in comparison to the other two. Their best day for speed, on average, was Black Friday at 5.4 seconds, which is not only worse than both Amazon and Walmart’s worst days, but it’s .2 seconds slower than their performance last year. Target’s slowest day on average was Cyber Monday, November 26 at 6.3 seconds, almost a full second slower than last year. Their fastest hour turned out to be on Black Friday at 5AM with 3.1 seconds, which is a slight improvement, with their slowest time being on Monday at 3PM with 8.9 seconds, over a second longer than last year, and sadly during mid-day on Cyber Monday.
(Target 8.5/10)

Geographic

It’s always interesting to see how sites perform differently across the world. If we look exclusively at the United States, it’s intriguing to see which states regularly see faster or slower times than others.

California has almost always come out on top as the fastest state, but this year they were consistently dethroned by none other than Oregon! For Amazon.com, the ecommerce mega-site saw average load times of 1.4 seconds in the The Beaver State, with their next-fastest location being Ohio at 1.6 seconds and Nevada at 1.8 seconds. When it came to their slowest locations, Washington, D.C. took the prize at a sluggish 7.5 seconds and Washington state clocking in at 7.3 seconds.
(Amazon 9/10)

Just like in 2017, Walmart.com was faster, but by a mere millisecond, seeing an average load time of 1.3 seconds in Oregon. Nevada and Ohio followed at Amazon’s fastest time, 1.4 seconds. Washington state saw the site’s slowest load time at 6.8 seconds, with Colorado coming in at 6.5 seconds and Texas at 6.3 seconds – all of them being faster than Amazon’s worst locations.
(Walmart 9.5/10)

Target actually saw some improvement this year with its average load time being fastest in Nevada at 2.3 seconds in (last year’s was 2.7 in California), while Oregon came in second at 2.5 seconds and Ohio third at 2.7 seconds. And like last year, Target’s fastest speeds proved to be slower than their competitors. The slowest average speed that Target saw in the U.S. was sadly worse than last year. Washington state clocked in at a truly dismal 10.7-second average load time, with Colorado a second behind at 9.6 seconds, and Texas at 9.3 seconds. It’s unfortunate that Target continues to miss the mark for website speed.
(Target 8.5/10)

Usability

For usability, we always select a common task a user might typically try to accomplish when visiting the sites we’re testing and replicate it. For last year’s Showdown, we decided to see what the experience would be like to use these three different websites to add a common product to the shopping cart. To do this, we selected one item to search for and add to our cart, and this year we decided to do the same again.

For each of these processes, we picked an easy item to search for, and sought to add a Blu-Ray copy of Disney and Pixar’s Incredibles 2 to our shopping cart. To begin each process, we started by opening a new tab in Google Chrome and typing in the site’s URL.

From the point of typing www.amazon.com into our Chrome browser, typing “Incredibles 2 blu-ray” into the store’s search box, and adding it to the cart, it took 34 seconds. From the front page, it took about 5 clicks (including having to log in to get to the final checkout) to get to the “Place your order” window.

From the point of typing www.walmart.com into Chrome and going through the same process, it took about 6 clicks and 32 seconds to log in and get to the final cart checkout page.

And from the point of typing www.target.com into our Chrome browser, it also took about 6 clicks and 32 seconds to log in and get to the checkout window.

Each site was a good experience to use, although each one has a different feel and approach. It’s a tough call to say which user experience we found to be better, but each one was straightforward and easy to use. If we judge the sites based on search results, Amazon tried suggesting a few things unrelated to the specific search of the “blu-ray” disc first (like a Jurassic Park daily deal and a preorder for Venom), while both Target and Walmart have more direct and accurate results (even though Walmart suggests the DVD and 4K before the actual blu-ray). In that case, we’d have to give Walmart and Target a little more props for accuracy in their product search.

(Amazon 9.5/10)
(Walmart 9.5/10)
(Target 10/10)

Verdict

With stakes this high once again, you would only expect the best from the leaders in ecommerce, so it comes as no surprise that the results were so good and so close.

With all things accounted for – reliability, speed, geographical performance, and the site’s usability – we’ve reached our verdict, and it surprises even us for a second year in a row:

Winner:

It’s that time of year again, where sales conscious bargain chasers brave the throngs of other sale hunters in the frigid November early morning air on that most dreaded of retail shopping days: BLACK FRIDAY. Just hours earlier, many of these same credit-card-wielding warriors were huddled around a table with family, giving thanks once again while stuffing themselves to their waistline’s discontent with mashed potatoes, roasted turkey and homemade pie. The juxtaposition of these two contradicting practices is staggering, but it’s no less the holiday tradition year after year.

As we approach another Christmas holiday, the world of ecommerce continues to ramp up the way they approach Black Friday–and its younger electronic sibling, Cyber Monday–with many now starting their sales right after Halloween. Accordingly, we decided to do something special for our next Website Showdown: a Black Friday / Cyber Monday edition that pits the ecommerce colossus Amazon against the websites for brick-and-mortar retail mega-stores Walmart and Target. It’s a truly epic battle royale to see how each site performs during the biggest shopping days of the year.

So, as usual, we used AlertBot’s external website monitoring system and its TrueBrowser™ technology to monitor all three sites from Thanksgiving through Black Friday and Cyber Monday, spanning from November 23, 2017 to November 27, 2017. We expected strong, reliable performance during the entire run and we were not disappointed. The results were nothing short of impressive.

Reliability

For the reliability evaluation of a website, we look for failure events (like when a page doesn’t fully load or it is completely down), and we look for what caused those failures.

Usually for this section, we evaluate each site’s performance in detail, drilling down to specific errors each one faced, and the different types of errors we usually see (like Slow Page Files, Timeouts, etc). It’s unusual for the sites in a two-site Showdown to not return a single error, much less a three-site Showdown. In this special evaluation of three sites, not one single, solitary error was found between them. All three sites avoided any kind of failure event or significant error. With the stakes so high for three of the biggest retailers on the most significant sale days of the year, one would expect nothing less. So, with that said, each site earns a perfect score for Reliability.

(Amazon 10/10)
(Walmart 10/10)
(Target 10/10)

Speed

When evaluating a website’s speed, we look at the time it takes the site’s homepage to render and load to the point of being fully interactive by the user. These tests are performed from the perspective of a first-time visitor with no prior cache of the website’s content. AlertBot runs the tests inside real Firefox web browsers using AlertBot’s TrueBrowser ™ monitoring.

Sites like Amazon, Walmart and Target boast very graphics-driven designs, and especially with monstrous sale event days like these, the graphics are often big, bold, and frequently changing.

With that said, of Amazon.com’s 5-day run, they saw the fastest day, on average, to be Sunday, November 26^th with 4.3 seconds. It’s not the slickest speed a site can have, but it’s certainly not bad. On their slowest day, on average, Amazon still clocked in at 5 seconds on Cyber Monday, which is still not too shabby. When looking at specific times of day for performance, the best hour was at 5AM on Sunday with an impressive 3.4 seconds, while Cyber Monday also saw the slowest hour at 7AM with 6.7 seconds.
(Amazon 9/10)

Walmart.com held their own surprisingly well during this time, too. Their best average day was Thanksgiving Day, November 23^rd at 4.2 seconds, just barely edging ahead of Amazon. Their worst day on average was Saturday, November 25^th, also at 5 seconds. Finally, their best hour on average was on Thanksgiving at a remarkable 2.7 seconds at 5PM. Their worst time on average was 6.4 seconds at 2AM on Sunday, November 26.
(Walmart 9.5/10)

Last, but certainly not least, Target.com didn’t perform quite as well as the other two, but they still performed respectably, especially considering the fact their site avoided any failure events. Their best day for speed, on average, was Thanksgiving Day at 5.2 seconds, which is worse than both Amazon and Walmart’s worst days. Target’s slowest day on average was Sunday, November 26 at 5.4 seconds, which at the very least, shows a great consistency for the performance of the retail chain’s online presence. Their fastest hour turned out to be on Black Friday at 9AM with 3.9 seconds, with their slowest being on Cyber Monday at 4AM with 7.6 seconds.
(Target 8.5/10)

Geographic

It’s always interesting to see how sites perform differently across the world. If we look exclusively at the United States, it’s intriguing to see which states regularly see faster or slower times than others.

California tends to prove to see the fastest web transaction speeds in the country, and in this test scenario, they once again come out on top for each website. For Amazon.com, the titan of ecommerce saw average load times of 2 seconds in the The Golden State, with their next-fastest location being Texas at 3.2 seconds. When it came to their slowest locations, Illinois came in at the bottom with 6.6 seconds, with Georgia just above them with 6.3 seconds.
(Amazon 9/10)

Walmart.com was only a millisecond faster, seeing an average load time of 1.9 seconds in California, also coming in faster in Texas at 2.7 seconds. But Walmart saw a placement swap for which state was the slowest, with Georgia coming in at the bottom at 6.6 seconds and Illinois right above them at 6.5 seconds.
(Walmart 9.5/10)

Target loaded on average at 2.7 seconds in California, with Texas coming in next at 3.5 seconds. Again, Target’s fastest speeds proved to be slower than their competitors. The slowest average speed that Target saw in the U.S. was also Georgia, at 7.2 seconds, but Washington stepped in as their second slowest, at 7 seconds flat.
(Target 8.5/10)

Usability

For usability, we always select a common task a user might typically try to accomplish when visiting the sites we’re testing and replicate it. For our previous Showdowns, we tested things like visiting a site for nutritional information or going through the motions of ordering movie tickets from a local theater. Like with the most recent Showdown for Lowes and Home Depot, we decided to see what the experience would be like to use these three different websites to add a common product to the shopping cart.

For each of these processes, let’s see about adding the PS4 version of new video game Star Wars: Battlefront II to our shopping cart. To begin each process, we started by opening a new tab in Google Chrome and typing in the site’s URL.

From the point of typing www.amazon.com into our Chrome browser, typing “Star Wars Battlefront 2” into the store’s search box and adding it to the cart, it took 30 seconds. From the front page, it took about 5 clicks (including selecting the autocomplete suggestion in the search bar) to get to the final “Place Order” window.

From the point of typing www.walmart.com into our Chrome browser, it took about 4 clicks and 35 seconds to get to the Cart Checkout window. The autocomplete was a little clumsy to deal with (it was tough to tell if the browser was really proceeding to load the site), but overall, it was a decent experience.

From the point of typing www.target.com into our Chrome browser, it took about 5 clicks and 27 seconds to get to the Cart Checkout window.

All three sites were good experiences, although each one has a very different feel. It’s a tough call to say which user experience we found to be better, so we decided to try a second test. This time, we chose something different, like Wonder Woman on Blu-Ray. We also decided to try Mozilla Firefox this time.

The process of finding the Blu-Ray disc and getting to the checkout process on Amazon took about 4 clicks and 25 seconds. The process on Walmart.com took 26 seconds and 5 clicks. On Target.com, it took roughly 24 seconds and 4 clicks. This time, we noticed that in the search results, there’s a convenient “Add to cart” option next to the items on Target’s site that Walmart and Amazon both DON’T have. This definitely gives Target a slight edge over their competitors. And with that being the only real significant difference, outside of its slightly faster completion time, we’ll have to say Target wins the Usability portion of this Showdown.

(Amazon 9.5/10)
(Walmart 9.5/10)
(Target 10/10)

Verdict

With stakes this high, you would only expect the best from the leaders in the retail industry, so it comes as no surprise that the results were so good and so close. This may be the toughest Showdown we’ve had to score yet, especially with three hats in the ring this time around.

But, with all things accounted for – reliability, speed, geographical performance, and the site’s usability – we’ve reached our verdict:

WINNER:

The report examined several sites that had experienced minor to substantial down times over the course of Friday thru Sunday, including NeimanMarcus.com and Staples.com. Our report, however, did not take into account Cyber Monday or the sales-heavy weeks that have followed.

On Cyber Monday, Neiman Marcus seemed to right the ship, while only a few sites appeared to experience no more than about 10-minute failures–including NewEgg.com and Staples–with Gap’s site registering 19 minutes of being unresponsive.

Throughout this month of December, Staples has continued to struggle with short outages; Walmart experienced some brief HTTP errors; while Footlocker, PC Mall, Crate & Barrel and Peapod all experienced just a couple few-minute hiccups (which in some cases could simply just be sluggish load times).

With online sales outperforming brick and mortar sales this Black Friday for the first time, it’s imperative now more than ever to make sure your site stays up and working, error-free. Sign up for a free 14-day trial with AlertBot.com today and start working towards a more successful ecommerce site!