#Benchmark standard
21 messages in this thread
I've noticed that some people are unsure of what the default settings are for
the chevy. So I was thinking that wouldn't it be possible for one of use to
make a chevy.vue (I think .vue is the file were you can render straight from
dos with specififc settings) file. It could the be uploaded and anyone could
download it and just render without having to worry about pixel size or
anything else.
Anyone else agree?
Just a Thought
Jason
Jason –
Hopefully everyone working on the chevy benchmark will see your suggestion. Try
to use REPLY to any message in a thread rather than WRITE or COMPOSE which
starts a new thread (conversation). That way using REPLY the whole talk stays
together as one unit.
Really interesting the different times members are getting.
Anne
Hi, Anne,
Amazing the response that's been rolling in, isn't it! Is compiling all the
responses into one document for the library something you could, would, or
would LIKE to do, or should I try?
We've gotten usable data on everything from my DX2/66 to P120s, and I really
think it'd be a pity not to compile it as the beginning of an on-going
reference.
SCL
Stephen, and everyone else – I read through the entire thread on the chevy.3ds
benchmark and don't think the data is readily usable. There are too many
inconsistancies in the data for it to mean anything. There are lots of people
rendering with mapping off and some with mapping on. Some people are rendering
to disk and others to display and yet others to null. Anyway, you get what I
mean. I was going to dissect the thread into the different times but it's just
not standardized enough. If someone else thinks differently and would like to
try and organize it then have at it. The thread wanders very little but there's
a ton of messages so it will take some time. If people want to continue with
this then lets define an simple standard. Like:
640x480x1.0 Pixel size 1.0 All defaults on except Mapping Render to Display
only Disk should be off
Any other ideas?
DM
Does anyone know if 3ds writes a .log file (or something) when rendering
from the command line? I know hardly anyone ever uses this feature… If it
writes a log file does it include the time it took to render?
Since it is possible to render from 3ds from outside of 3ds using command
line switches we could create several batch files that control all aspects of
the rendering and anyone could download these batch files. Then post the .log
file as results which could then be complied later by a voluntier into a
cohesive document.
Another simpler solution?
Doesn't saving out a PRJ file save all info including rending paramaters?
Let's make several Chevy.PRJs (alias bnchmark.prj) and put them out here for
users to download and render.
When I have some spare time <G> I will look into this myself.
Q. What part of the window are you looking at to get your rendering time? My
render dialog box time sometimes varies from the time shown in the status bar.
Q. What Release? Is there a difference between the rendering engine from 3ds3
and 3ds4? I'm using r3 my next upgrade will be MAX not r4.
Q. What about a separate, bigger test model to put faster computers "through
their paces"? I had a rendering time of 0:20 w/ no mapping at 640!!!
Timothy
NVision Inc. & Digibotics
Dave,
I really think you are right. I had some good times(I think), but I would hate
to stand up in front of everyone and proclaim that I had the "x" fastest time.
There's no way to know for sure, just too many questions about the data.
I would suggest that the resolution be 1280×1024, this would make the times
longer and any 1 second difference would be a smaller percent increase/decrease
in time. If one time was 20sec and another was 21sec(which is possible with a
P133 & no mapping), then the 20sec system would be 5% faster. Even 1 second out
of 60 is 1.6% faster, it seems to me we would need a minimum render time of 2
minutes for a P133. We have to consider that the P6 is coming and then and
then…. For these times to be useful in the future we need some longer times
now.
I would like to see times with and without mapping. It depends on what you want
from the numbers, CPU test or system test. I think both would be nice.
Regards………….Rick – Technical Animations
Dave,
Yeah, I'd pretty much come to the same conclusion about the variable variables.
I agree with most of what Rick Miller says, about creating a .PRJ to d/l for
those who want to participate, with but one quibble.
I agree that we need a longer render time now, so that down the road, rounding
to whole seconds doesn't become a bigger error factor than it already is.
Just for the fun of it, I took a bunch of numbers for relative processor speeds
that have been floating around and (with the dubious assumption that MAX for
Alpha code would render at the same rate as R4 code) came up with approx. 14
secs. for Chevy @ 640, mapping on, on a dual 300MHz Alpha setup. This figuring
has a lotta holes in it, but it's defensible. (I'll elaborate if you're
interested…) I mention this to show what we'd be up against if we wanted to
put together a single frame that took 1 min. to render on this setup, without
the use of slow IPASs like Vapor.
But getting it with a higher resolution than 800×600 is going to keep anyone
with a 2-meg video card from participating.
The reason I'd come up with 640×480 was to include 16-meg machines w/o a swap
file, but as it turned out, I only recall one respondent with 16 megs. Lots of
people, however, are using 2-meg video cards.
A possible better solution (since we're already talking about a special .PRJ)
would be to just put more stuff in the scene…
I thought of using PCACITY.3ds off the WCT, but even at 640, it creates a 7-1/2
meg swap on my 32-meg machine. Since so many of the respondents had 32 megs,
it's probably too big.
Maybe somebody else can chime in…
SCL
Steve,
I think the benchmark chevy.3ds we have is fine – except resolution. These
parameters were established long ago by the tech support guys at Adesk.
To avoid problems with display cards it has always been said that the "no
display" button should be active (same effect as rendering to NULL device).
I suggested sometime ago to have mapping ON, then OFF just to eliminate or
delineate differences in disk sub-systems and cache schemes.
Because systems are getting faster and faster and times shorter, I suggest the
benchmark be raised in resolution and times shown with/without mapping, no
display, anti-aliasing ON, pixel size 1.1 (the shipping default).
Somehow we need to keep the rendering time above 100 seconds in order to see at
least whole percent speed differences.
Martin,
You're right, I'd forgotten for the moment that rendering to NULL will keep it
from defaulting to Render Display Size.
Didn't know that Chevy.3DS was a "quasi-official" benchmark, though. Probably
1280×1024, Metal, Alpha on and a Pixel Size of 1.5 would give us a long-enough
render time to be useful.
Why don't you try it on that Onyx, to give us all an idea what a (practically)
cost-no-object machine can do with it? I'll do in on my poor old 486/66 when I
decide I need an hour off <g> to give us an idea what the other end of the
spectrum looks like. (Hope it'll fit in my 32 megs o' RAM!)
From there we can all evolve a consensus on how to configure one .PRJ so we
have something statistically useful.
SCL
Stephen,
Those settings look good to me, but I better render it with those setting and
let you know.
Regards………..Rick – Technical Animations
I thought that was what I said? 🙁 Anyway, several test resolutions are not
only practical but a must. Most animation is not rendered to 1280×1024 (esp.
around here) and I'm not going to wait for a 1280×1024 rendering on my 486.
Timothy
NVision Inc. & Digibotics
BTW …I have a 100MGHZ going 0:20. No doubts, I am a Veteran PC/SGI
Modeler/Animator/Sculptor. If you need an object created and/or digitized on a
3D laser scanner drop me a line.
Timothy,
Oops, sorry; credit where credit is due, and all that. I think I was going on
my 16th hour straight at the keyboard when I wrote that — the 4:04 time on the
offending message was A.M.! (Been modelling my a** off!)
I'm not entirely sure about this, but I _think_ the reason you see the
different rendering times is that the screen-bottom line and time in the status
pull-down both include the time it took to load the maps, but the last-frame
time in the render dialog box doesn't.
Once again not sure, but to the best of my knowledge the render engine didn't
change from R3 to R4 (Part of the reason they were calling R4 the plug-in
upgrade.)
I'm going to try 1280×1024, Metal, Alpha on, Pixel Size 1.5, on my 486/66 to
see how slow it is… I think that having multiple .PRJs would only lead to
confusion and negate half of the reason for doing this to begin with.
BTW, I see two main reasons:
(1) To compare relative times of different CPUs
(2) So those interested can compare their times with others' in the same CPU
class.
If you'd rather not try it on your 486, that's OK — we really don't need to go
hog-wild in compiling stats on the old stuff… But a few'd be good for
comparison purposes. I keep hoping somebody that still has a functioning 386
will try it for the sake of the archive, so 10 years from now we can look back
(probably on CIS-VR by then <g>). The manual says R4'll run on a 386 if you've
got 16 megs.
Well, back to the mines.
Let me know what you think about this…
SCL
Hi Stephen,
>>Oops, sorry; credit where credit is due, and all that. I think I was going on
my 16th hour straight at the keyboard when I wrote that — the 4:04 time on the
offending message was A.M.! (Been modeling my a** off!)
Thats OK, It's just that lately I have been contemplating whether people who
aren't on-line exist or not and felt my own existence slipping away. I guess I
don't get out much. 🙂 16hrs 4:04! sounds like yet another subtle form of
socially acceptable torture. Of course for myself I find that I am the most
creative at 4am. Or is it just that at 4am I "Think" I am?
>>I think that having multiple .PRJs would only lead to confusion and negate
half of the reason for doing this to begin with.
Its just that many of my peers are still chugging along on 486's…Quite nicely
as far as they are concerned.
I think that this is a great idea that will be taken very seriously by
everyone. The people in the Bench Mark forum will be impressed. This will be
very very useful to anyone contemplating on buying a PC for 3ds. There aren't
enough benchmarks for 3D and esp. 3d practical apps. I am willing to help.
If we also post all of our results to the Sysop Rick Harris in the benchmark
forum I'll bet that they will compile the data for us ( they are starving for
real 3d benchmarks). Takes most the work outta it. Or we can do it. Any
opinions?
Timothy
Timothy,
Doesn't bother me a bit to pass along our results, raw data or compiled —
science knows no national boundaries, or whatever that quote was…
Part of the reason why I've been rooting to get this done is precisely because
the mainstream publications really don't address our particular interests in
looking for the best systems. Sure, you can get an idea by comparing Dosmarks
and FPU performance, but that's not the whole story.
Actual rendering times are, to my mind, all that really counts.
For the time being at least, you can count me among those still chugging along
on a 486, though I'm not happy with it any more and am desperately hoping that
Intel releases P150s at or just before Comex.
I'm not too certain that I do my 'best' work in the wee small hours, but I do
know that there are a lot fewer distractions, and for me at least, the more
complex modelling that I keep tending towards demands focussed concentration.
Anyway, I've got a full plate today, so I'd best make like the proctologist and
get crackin'!
SCL
Stephen,
>>I agree with most of what Rick Miller says, about creating a .PRJ to d/l for
those who want to participate, with but one quibble.<<
Actually I think you are refering to Timothy Smith, but if you insist I will
take the credit<BG>.
>>But getting it with a higher resolution than 800×600 is going to keep anyone
with a 2-meg video card from participating. <<
Is there some reason we can't render to a Null Device, we don't really need to
look at it do we? This certainly is a small point though, there are other ways
to increase render times. Turn on alpha, increase pixel size or duplicate the
car or some object in the scene. What would you suggest?
Regards……………..Rick – Technical Animations
Rick,
Somehow in my blurry, 4 a.m. mind, those two msgs got confused or rolled into
one or something…
You're right, I'd forgotten for the moment that rendering to NULL will keep it
from defaulting to Render Display Size.
Anyway, see my msg to Martin for a proposal for a standard .PRJ, and let me
know what you think…
SCL
Rick,
I think that using the Chevy.3ds that everyone has is the way to go. If
everyone agrees I believe we should use the following settings to begin with.
Martin's right about having to keep times over 100 sec. If we get times faster
than this we'll have to change something.
How about: 1280×1024 Metal, Mapping on and off. No display. Pixel size 1.1
How about disk? I suggest leaving it off.
Stephen,
I rendered the Chevy.3ds file again with our new settings for our benchmark
standard, this is what I got.
SuperMicro P55CMS P100 w/Pipelined Burst Cache, 32mb ram, 60ns ram, 2.1GB Fast
SCSI HD, Adaptec 2940w SCSI controller.
1280×1024, Metal, Null Device, Pixel Size=1.5, Didn't Save the File, 3DSr4c1
Mapping Off = 3:22
Mapping On = 4:45
Turning Alpha on does not affect the times. I also tried the above settings at
1024×768 with the mapping off and got 2:16. I think that resolution is going to
give times that are just too short for the P133, Alphas and the future. Even
the 3:22 might be too short when you consider the future 24 months from now.
Regards…………….Rick – Technical Animations
Rick,
What do you think we should do about the .PRJ? I'm thinking maybe selecting
all, then shift-mirroring selected, and pulling the camera back and/or opening
up FOV to get it all in… Probably won't quite double the time, but ought to
add a couple of minutes, anyway…
You've probably seen my dual-Alpha estimate; I think if we can arrive at a .PRJ
that yields close to 100 sec using my calculations, it's gonna be about as good
as we can get now. Beyond that, I fear, we'd be trying to estimate Pentium
speeds on a 286! And after all, if some future quad P7 does the render in :10,
we'll be so far beyond what we're doing now that these times will be little
more than a source of amusement, anyway!
Whaddaya think?
SCL
Stephen,
How long do you think the Pentium render times will be when the chevy file is
big enough for the Alphas. We don't want to make it long for the Pentium, some
people may not do the render for the benchmark and send the times in.
AT 1280×1024 my times were 3:22 with mapping off and 4:45 with mapping on, will
this not be long enough the Alphas?
Regards……………Rick – Technical Animations
Rick,
Actually, I think it's gonna have to be long enough… (See related exchange w/
Greg P.) FWIW, a 50-sec. rendering will give us 2% accuracy, which is probably
plenty good enough, and my Alpha estimates are probably close enough that a
dual 300MHz box is still going to take longer than that at 1280, metal, mapping
on, Pixel Size 1.5, no save, no display.
I've got a sub-dir in my File Cabinet labelled Benchmark, and your most recent
times are in it. Reality being what it is, and this not paying any bills as
someone remarked a few weeks ago, I suppose we'd best be satisfied with what we
can get.
Now if we can just get the level of response we did the first time around…
SCL