VisorCentral.com - Are Benchmark's Accurate

Show 20 posts from this thread on one page

VisorCentral.com (http://discussion.visorcentral.com/vcforum/index.php)
- Visor General Chat (http://discussion.visorcentral.com/vcforum/forumdisplay.php?forumid=31)
-- Are Benchmark's Accurate (http://discussion.visorcentral.com/vcforum/showthread.php?threadid=19760)

Posted by dampeoples on 11-20-2001 03:19 AM:

Are Benchmark's Accurate

I ran a benchmark program I got somewhere just for kicks the other day. I have a Prism, which should be faster than the VDx. The speed for a VDx was 21, the same as my Prism, how accurate are these things?
By the way the name of the program was Speedy.
Sterling

Posted by dkessler on 11-20-2001 02:38 PM:

Re: Are Benchmark's Accurate

quote:
Originally posted by dampeoples
I ran a benchmark program I got somewhere just for kicks the other day. I have a Prism, which should be faster than the VDx. The speed for a VDx was 21, the same as my Prism, how accurate are these things?

By the way the name of the program was Speedy.

Sterling

Perhaps a better way to phrase the question is "are benchmarks meaningful"

Writing good benchmarks is not easy. Most benchmarks are "synthetic" - meaning that they perform a contrived sequence of operations as opposed to actually timing tasks in real applications. The operations a benchmark measures must be carefully selected by the author to give meaningful indications of particular aspects of system performance. But there are a lot of different things that can be tested such as database access, computations, graphics speed, text speed, form operations, etc. and figuring out how each of these factor in to "overall" system performance is tricky. To make matters works, performance aspects that are critical to some apps may not be nearly as important to others.

For those reasons it's important for benchmarks to explain what they are testing and how they arrive at their overall numbers. For example, Kopsis' VFSMark benchmark utility shows benchmark scores for all the individual test components and tells how they all factor into the overall result. On top of that, we provide the source code for the benchmark app so that other developers can analyze what our tests are doing and tell us if there are areas that need improvement.

If a benchmark doesn't tell you what it's testing or how it's getting its numbers, and it doesn't provide source code, dump it

There are plenty of good ones out there that do.

__________________
<ul><li>Dave Kessler<br>President - Kopsis, Inc.</li></ul>

Posted by bradhaak on 11-20-2001 04:07 PM:

Exactly correct

As Dave says, benchmarks are generally accurate about what they are testing, but you have to be able to evaluate how meaningful that is to your application.

All times are GMT. The time now is 12:23 AM.

Show 20 posts from this thread on one page