PCMark 8 – The Most Important System Benchmark in the World
8/23/2013 by: Anshel Sag
We have been working with the guys from Futuremark for many years and they have consistently provided solid and reliable benchmarks for as long as we can remember. PCMark is one of their benchmarks that has been around for quite some time and has been the company’s attempt to benchmark the entire whole system’s performance rather than just focusing on the GPU. Futuremark is actually better known for their cross-platform 3D benchmark 3DMark, which is by far one of the best and most consistent 3D benchmarks ever.
The company has been putting out the PCMark benchmark for more than 10 years since PCMark2002, which obviously debuted in 2002. For the last 11 years, Futuremark has been learning about how to make a better whole system benchmark and they’ve slowly made perfections upon their design and philosophy. As they’ve grown as a company, so have their benchmarks and PCMark is definitely something to behold. This isn’t their only popular benchmark, since the company has been around since 1997 and has shipped over 100 million copies of their benchmarks and has resulted in over 40 million results that are captured within their databse.
PCMark 8 has been around for long enough and we have been working with the company to understand how important the benchmark really is. It is always used by BSN* whenever we test a whole system like a laptop or a desktop and we consider it to be an ever increasing measure of overall user experience. This is primarily because of the company’s focus on the need to benchmark real applications and real user scenarios, which include things from SSD speed tests to browser benchmarking (which Futuremark also does with their Peacekeeper benchmark.
This version of their benchmark puts a renewed focus on things like OpenCL GPU compute as well as web browsing performance, which are both fields of growth within the PC market. PCMark 8 is consisted of Home, Creative, Work, Storage, Applications and Battery life tests. All 6 tests are designed to properly address the needs of different types of users with different workloads and answer certain questions like the real-world performance of storage solutions as well as the real battery life of devices. And obviously, PCMark 8 is built with Windows 8 in mind since PCMark 8 can only currently be run on Windows, however, I do believe that Futuremark will likely push PCMark into mobile like they have 3DMark.
Most people’s exposure to PCMark 8 is primarily when they look at the benchmark when they read a review like the ones on our site. However, most people don’t that almost all press get PCMark 8 for free as a part of Futuremark’s press program to enable the press to use better measuring tools. In addition to the press versions, Futuremark has fully featured professional versions and retail versions. They also have a free version, which allows for limited use, but is equally as informative as the press and professional versions in letting someone know how their system is performing. It also lets you know if your system is performing up to expectations when the scores are uploaded to Futuremark’s database. Futuremark’s database is probably one of the best repositories of hardware performance in the world because of the multitude of benchmarks and years of use. The majority of Futuremark’s actual revenue and sales still primarily come from the professional benchmarks and support. This helps a multitude of companies understand the performance of hardware and make the appropriate decisions necessary based on PCMark benchmark results.
We spoke with Oliver Baltuch, Futuremark’s President, and got the low-down of exactly who is using Futuremark and why. He gave us a broad understanding of PCMark’s role in the PC industry, one that we simply didn’t realize existed. Sure, PC OEMs like Dell, HP and Lenovo (not officially stated as customers) use PCMark for their Desktop and Notebook testing, but the list is much much longer. Oliver explained that many governments use Futuremark’s PCMark benchmark for testing, including our very own government at multiple levels. Not only do federal and state governments use PCMark for their testing, but local governments as well. Such a vast use of PCMark in governments is also found in other countries outside of the US as well. The aerospace industry as well as insurance, medical and financial industries also us PCMark. If that list wasn’t long enough, add automotive and heavy truck manufacturing to the list, as well as Heavy Industry, oil and gas and telecom companies. But that’s not it, commercial and industrial software companies, OEMs and integrated hardware vendors also use PCMark as well as electronics retailers, who obviously want to sell the fastest products on the market.
Futuremark also has a BDP program which enables virtually everyone that uses the program professionally to provide feedback and get support in making the benchmark as transparent and fair. To ensure the utmost level of transparency, the BDP program allows its members to preview the benchmarks months before the benchmark’s release to make sure they are able to iron out any issues ahead of time. Joining the BDP program requires a fee, which is one of the ways that the company makes money including professional licenses and consumer licenses. Some BDP members include companies like AMD, Dell, HP, Intel, Lenovo, NVIDIA, Qualcomm and Samsung.
Since CPUs and APUs are the focal point of any PC, we took a look at how some of the latest processors stack up against each other and found some interesting performance to price numbers comparing the average PCMark scores of certain processors against retail prices on Newegg.com.
The hardware used for testing is as follows:
APU systems using 8GB (2x4) GB DDR3-1866 (AMD A10 6800K and 5800K using DDR3-2133, All remaining systems used DDR3-1600), Windows 8 64bit, Driver 13.101
Intel systems using 8GB (2x4) GB DDR3-1600, Windows 8 64bit, Driver 22.214.171.124.3071
Taking a look at this graph, you can see that AMD’s OpenCL gives them a slight edge of Intel’s CPUs when you take into consideration Futuremark’s decision to give OpenCL performance more weight. Additionally, Intel’s raw CPU performance is taken less into consideration in this benchmark since it may not always result in a better user experience when other compute functions are being utilized. If you look at the price to performance ratio, you can also see that both Intel’s and AMD’s CPU’s pretty much reach a point of diminishing marginal return as performance increases with the gap being narrowest at the lowest price and performance. AMD still delivers some impressive price to performance when you simply compare the red against the blue bars.
Considering that OpenCL is being utilized in applications ranging from Handbrake, FCP and Adobe’s Photoshop, it doesn’t seem that drastic to understand Futuremark’s decision to give OpenCL more weight, especially with Khronos’ own standards gaining more and more favor with every new version. Not only do productivity applications utilize OpenCL, but digital entertainment software does as well with apps like VLC player and Battlefield 3 (and likely Battlefield 4) being a few among many.
There is no hiding the fact that we’ve been proponents of Khronos’ own open standards like OpenCL, but it is quite obvious that developers want it and that it will be the cross-platform API for GPU computer on both mobile and PC. I just want wait for PCMark to reach mobile so that we can start comparing mobile and PC system experiences and battery life like we do GPU performance in 3DMark.
We look forward to continue using PCMark 8 in our laptop and desktop systems continuously until a new version comes out and hope and pray that a mobile version comes out to help decipher the crazy world of mobile. With versions for iOS, Android, Windows Phone, Windows RT and other operating systems, we can finally have the holy grail of performance benchmarks.
Futuremark, PCMark, PCMark 8, GPGPU, OpenCL, Khronos, GPU, Compute, Web
© 2009 - 2011 Bright Side Of News*, All rights reserved.