[Mihai]

We haven't abandoned Benchwell, far from it - its integration with Maxwell will be much improved
But there is one crucial component missing - a scene.

So we are asking for your help to make a collaborative project to create the next Benchwell scene. Just a few constraints:

It can't be gigabytes in size of course and it should render more or less clean in about 15 minutes on a common 4 core system.
And also, it has to look cool

It could use many of the new features in 2.6 also.

Lets start the brainstorming....

[numerobis]

Nice!

... its integration with Maxwell will be much improved

but wouldn't it be better to have a stand alone app like cinebench with only a "start" button?
With an automatically generated chart containing the result and the hardware specs and with the ability to send it to the database if you are registered.
A closed one touch system.
I think then this could become a standart test app for many hardware review sites.

[Mihai]

Well, hence the " "

[numerobis]

[dmeyer]

Here is Adrian's idea from a couple years ago:

http://maxwellrender.com/forum/viewtopi ... =benchwell

[m-Que]

Awesome!
Also great idea for letting the community to participate!

but wouldn't it be better to have a stand alone app like cinebench with only a "start" button?
With an automatically generated chart containing the result and the hardware specs and with the ability to send it to the database if you are registered.
A closed one touch system.
I think then this could become a standart test app for many hardware review sites.

Yes..., but we will STILL need that scene, one way or another

[roch_fr]

I think that we need for this scene to disable the saving of the MXI in order to test only the proc. performance and to avoid the HD/SSD writing time.

[Mihai]

Put your artistic sombreros on, these technicalities are easy to deal with once we have a cool scene

Just some ideas for scenes:

A mad scientist lab setting
RGB "paints" flying around, hitting someone or something (we can use RealFlow for this)

Should it look clean/photo product, or used/realistic/amateur photo?

[RenderFred]

Some suggestions:

I believe the scene should reflect (ah ah) what people use Maxwell for. Of course people use Maxwell for everything, but I believe that a complex architectural scene (interior) would be perfect. Lots of objects, lots of different materials, lots of possibilities. I believe the scene should have a big Multilight MXI (at least 1 GB) so that people using networks and renderfarms could also have a way to measure the merging time, which is definitely not negligible when you render on more than one system. Of course it should also be possible to render the scene with the NoMXI flag to éliminate the MXI writing time when running on a single system. My point is: instead of having a simple scene which is not representative of what Maxwell is used for, use a real life scene.

Now the ideal thing would be to have two scenes: a simple one, benchwell-like, fast and easy to render (kind of Cinebench), and a big one for people who don't care about bragging rights and want to see something more serious.

[polynurb]

as fred says, the old benchwell has shown that it does not fit all purposes/machines.

maybe for a farm/network test the resolution could be simply be raised?
eg. 960x600 for single coputer 2560x1600 for farm/network?

the average time it takes to complete should not be too long, as people simply won't use the benchmark much if it takes 1h on a 2600k.. so i am afraid we have a certain limit of complexity that can pe put into the new scene.

[oz42]

I agree with polynurb, the scene should render pretty quickly. I also think it should be very simple to download and use.

A lot of CPU reviews still include the BenchWell results (which I find very, very useful for comparative purposes) and I think they use this not only because BenchWell is a very good test of processor intensive tasks but also because it is quick and simple. I think the new Benchwell would fall out of favour with reviewers if it was hundreds of MB to download, many settings to get it going and took hours to complete even on high end systems.

I think that we could do with a more 'real world' benchmark but I wouldn't make it so real world that people (reviewers) stop using it. Maybe one solution would be to have a couple of different test; a short version for most people and hardware reviewers (BenchWell) and a more compleX version (BenchWell-X) to really test out more realistic situations.

[Half Life]

You can create a scene that is both geometrically very light that at the same time will render slowly... often IBL/Image-based emitter materials is a big culprit in total download size of a scene due to high bit-depth.

An interior scene with standard emitters and some SSS materials in there should be sufficient to slow it down enough for there to be sizable differences between systems -- which is what I would assume you would want from a benchmarking tool... I get so tired of seeing benchmarks with minuscule time differences.

I'm thinking I would want to set it to render to higher sample levels (18-20) so that you could really gauge performance.

I would throw content ideas out there but the reality is: what the scene is of really matters much less than what it measures (which has nothing to do with how it looks).

Best,
Jason.

[limbus]

Hey there,
I think its a great idea to create a new Benchwell scene. I will post a scene that might be suitable for this. I just need to replace 2 commercial textures with free ones.

Can we use the Arroway textures that come with maxwell in this scene?

Its a geometrically simple bathroom scene. Someone could add a SSS soap bar and maybe some Realflow water to it to make it a little more complex and render intensive.

I think we could and should aim a some higher render times than the usual benchmark scenes. This is maxwell after all and computers are not getting any slower. Maybe 20 minutes on a newer 8 Core machine for a clean render. Another way to have a faster benchmark would be to let the scene render for a given time and then take the SL as the benchmark.

Cheers, Florian

[brodie_geers]

I think the problem with having a fixed time and variable SL is the sort of exponential time each SL takes. You'd need some sort of conversion chart for comparing a computer with a score of SL21.2 vs. SL20.2 which is a far greater difference than SL14 vs. SL10. What's nice about the "current" method is that if a computer scores 2minutes you know it renders twice as fast as a computer which scores 4 minutes.

-Brodie