Page 1 of 3

Network Rendering...

Posted: Thu Jul 06, 2006 12:12 am
by Mr. Gog
Working or Not?
If yes...let's start a thread with instructions.
Thanks.

Posted: Thu Jul 06, 2006 7:57 pm
by deadalvs
didn't try yet !

but i doubt that it works on macs alone. if, then it works as i think only as server with a pc manager...

but let me try this with the pc.

* * *

deadalvs

Posted: Thu Jul 06, 2006 8:24 pm
by Mr. Gog
I tried only with macs, but it did not work.

Tell me if it works wih a pc as manager. Maybe it can work with an intel mac running windows xp. I'am considering that.

Thanks.

Posted: Fri Jul 07, 2006 4:50 am
by kirkt
I tried with one intel mac (MacBook Pro) and one PowerPC mac (G4). mxcl started up in server mode and manager mode no problem and the machines immediately recognized each other. I started mxcl-d on the server machine and the maxwell window came up no problem. However, I could not select a .MXS file when I tried to add a job - I could get the file dialog to come up, i could navigate to the .MXS file but when I chose the file and dismissed the dialog, the file was not entered into the add job window. An error on the manager machine said that there was no active camera in the .mxs file - I was trying to add Thomas An's material testing file.

I found that if I lauched mxcl from within studio (by hitting the "R" render button) instead of mxcl -d, mxcl would open and begin to render as usual. I stopped that render, went to the network tab and then, when I added the job, the .mxs file that I sent from Studio was already entered into the add job window. However, when I added the job, checked the "Cooperative" box and hit "add", the network render began but immediately failed.

In short - it did not work. In the manual it says that the computer names cannot contain spaces or other illegal characters - I wonder if this is the issue.... So, how to change my computer's name without completely messing things up....

Posted: Fri Jul 07, 2006 1:38 pm
by deadalvs
could there be a problem with the path length in characters?

this had been a problem with other software...

* * *

deadalvs

Posted: Fri Jul 07, 2006 4:36 pm
by Mr. Gog
I set up everything, the manager and the servers, they see each other. If you open an mxs on viewer, load mxs on file menu, then you add the job, there is no problem.

But in the renderfarm part i don't see any machine, and then if i add the machines by it's ip the say in red: "disconected". They don't start the render, there are no problems with the mxs file.

Were's the trick. Someone from NL can say something.

Posted: Fri Jul 07, 2006 4:48 pm
by kirkt
I actually just got a job to render over my office network, between 2 machines. I then realized that one of the machines was running 1.0 and the other was running 1.1 - I am updating to 1.1 and will rerun my MacGuyver kluge job and post how I did it as soon as I can claim success. Things I have noticed so far:

1) the info in the network panel regarding SL and all that does not appear to update, at least in my minimal (one time) successful trials.

2) I'm not sure you have to do this, but one of the machines that I designated Manager via the terminal ("./mxcl -manager") I also designated a server ("./mxcl -server") in another shell. Then both machines show up in the connect list and get the render job. When I had one machine designated server and the other designated manager, only the server got the job and rendered. Use the "Activity Viewer" app, or the shell command "top" to check the processor activity of each machine to verify that it is even doing anything.

3) You have to set your render output to A NETWORK DRIVE!!! This way all of the machines can see and assemble the files in a commonly acessible location. This may seem obvious to some, but not to me.

4) As i posted above, I was having problems getting the MXS to add - I solved this as previously posted by Mr. Gog and others by using File>Open MXS within mxcl. Problem solved.

I will try again with the 1.1 update on the older machine and try to give a simple explanation of my steps once I actually get the thing to work again after the update.

Cross your fingers!

kirk

Posted: Fri Jul 07, 2006 5:56 pm
by mgroeteke
Mr. Gog wrote:I set up everything, the manager and the servers, they see each other. If you open an mxs on viewer, load mxs on file menu, then you add the job, there is no problem.

But in the renderfarm part i don't see any machine, and then if i add the machines by it's ip the say in red: "disconected". They don't start the render, there are no problems with the mxs file.

Were's the trick. Someone from NL can say something.
did you set the firwall correctly? you have to open port 51914 on all participating macs.

Posted: Fri Jul 07, 2006 6:15 pm
by kirkt
Well, apparently I have been successful. I had trouble getting Thomas An's scene to work, but that was because I was too lazy to re-download it to my manager computer. I will try getting that to run next. I ran an old RC5 scene, so ignore the materials, lighting, etc. The basic idea is that it freaking worked. I am amazed - I have a knack for brute forcing my way through this stuff, usually ending with hangs, crashes and smoke.

Here is my test.

Mac Rendering Network

1) G4 dual 1.0 GHz PowerPC, using V 1.1 (original license)
2) MacBook Pro, Intel Core Duo 2.16 GHz, using V 1.1 (plusone license)

Connected over the office intranet. We have PC/Microsoft-based servers and networking/network volumes.

The G4 desktop (represented below by the "$" prompt) was the manager, with the file to be rendered on a local drive for that machine. This machine was also designated a server. This is the machine that I used to launch mxcl -d ("Viewer") to open the MXS and add the job.

The MacBook Pro (represented below by the "%" prompt) was a server.

A network drive was used for the output (test.tga) and the mxi (test.mxi).

The below assumes you have navigated to the directory containing your "mxcl" app typically contained in

/Applications/Maxwell/maxwell.app/Contents/MacOS

I have not added mxcl to my PATH environment variable yet.

It is advised that you use Activity Viewer (Applications> Utilities> Activity Viewer.app) or the shell command "top" to check your process and processor activity.

Step-by-step approach:

1) Open a terminal shell and start the server on the MacBook Pro

% ./mxcl -server

-- you should see the terminal window give you feedback that mxcl has started in server mode

2) open a terminal shell and start the manager on the G4

$ ./mxcl -manager

-- you should see the terminal window give you feedback that mxcl has started in manager mode

3) In a new shell start the server on the G4

$ ./mxcl -server

-- you should see the terminal window give you feedback that mxcl has started in server mode

4) Start the Viewer on the G4

$./mxcl -d

-- the typical viewer GUI should open. This is where you will be doing the "Add Job" step later.

5) Start the Viewer on the MacBook Pro
EDIT: After further testing, this step appears unnecessary

% ./mxcl -d

-- the typical viewer GUI should open.

6) In the Viewer app on the MacBook Pro, click on the "Connect" button in the Network tab panel.
EDIT: After further testing, this step appears unnecessary

-- You will be connected if all goes well and you should see both of your machine IP addresses in the "Render Farm" list, with the "Connected" status in green. So far, so good.

7) In the Viewer app on the G4, click on the "Connect" button in the Network tab panel.

-- You will be connected if all goes well and you should see both of your machine IP addresses in the "Render Farm" list, with the "Connected" status in green. So far, so good. Some times I got a crazy sequence of "Connected/Disconnected" messages in an infinite loop - the shell window said something about the Local Host refusing the connection. If this happens, try quitting the Viewer and relaunching it and repeating Step 7 (use the bang [!] or up arrow in the shell for a quick repeat of the previous command in that shell window [./mxcl -d]).

Eight) Now, in the Viewer on the G4 (the manager machine) choose:

File >> Open MXS and navigate to the local MXS file you wish to Net Render. Select and open it. Incidentally, the actual digit "eight" followed by a close parenthesis makes a smiley like this 8).

9) In the "Render Options" Tab of the Viewer, make sure you set your options, most importantly the Output Image Settings and the MXI settings - set these to a NETWORK DRIVE. I have no idea if both or either actually NEED to be set to a netwrok location - I will leave that as an exercise for you.

10) In the Viewer on the G4 (manager machine) go to the Network Tab and choose "Add Job". The dialog that comes up should already have the MXS file you just opened in the "MXS File" input field and all of the Command Line options should be there as well. IMPORTANT: Tick the "Cooperative" Checkbox.

11) Hit the "OK" button and pray.

-- You should see the job listed in the "Jobs" Window - if you click on the expander "+" next to the job, you can see each node of the render farm doing its thing. The SL did not update for my job. The "State" field will update to tell you about what is going on. Essentially each node will render (you can see what is happening in the shell where you gave the "./mxcl -server" command) and then wait to assemble the MXI and final image. Both of these files are output to the netwrok drive location you specified in Step 9.

Once all the nodes are finished the final image will be generated and you should get the state of each node updated to "Success". Congratulate yourself and go have a cold one.

Here are some screengrabs:

Image
"I can't believe it is actually rendering!!!!"




Image
"SUCCESS?! Are you kidding me? And I didn't even break anything?"




Image
Activity Viewer of the Manager/Server G4





Image
Activity Viewer of the Server MacBook Pro


Have fun - I hope this works for you. I probably have needless steps in there, but give it a shot and experiment. I'll keep trying to figure it out. Anyone with more than 2 machines should try to see what kinks occur with multiple nodes. If you have a Mac network and got it rendering in no time, shame on you for not speaking up sooner and leaving it to an idiot like me to publish instructions that will probably crash everyone's machines. If you do crash following these instructions, I apologize - like I said, I have a knack for crashing and hanging computers.

CTRL-C is your friend.

kirk

Posted: Fri Jul 07, 2006 6:25 pm
by mgroeteke
one question: did you do cooperative rendering? interesting to see that you use the -d option on both macs. is there any reason for this? ...and what happens if i don't set a network drive as output location?

cheers

markus

Posted: Fri Jul 07, 2006 6:28 pm
by kirkt
I did cooperative rendering and I did use -d on both machines. i figured the more feedback the better. The only reason i used it on the SERVER ONLY machine (The MacBook Pro) was to hit the "CONNECT" button. I am not sure this is necessary, but I haven;t had a chance to test that yet. Both machines used two processors until they were finished. The manual says you can open the Viewer on any SERVER and add jobs, but I did not have success attempting this on the MacBook Pro (SERVER ONLY) machine. I am not sure if this is pilot error on my part, or an actual error that leads me to believe that you have to add jobs fom the manager machine. I have to use a benchmarked, timed rendering on a single machine and then test that MXS in the networked farm to get an idea of what is really going on, another test to do!

Again, another test to be done.

kirk

Posted: Fri Jul 07, 2006 6:39 pm
by kirkt
I just tried sending the same successful job but with a single change: I changed the location of the saved image to the manager/server (G4) local drive. I saved the MXI to the same network drive as before.

I started the render and I got this error:

Memory error: opening targa file>> Error Writing File: /Users/kthibault/Desktop/License/non_net_drive_out_Net_MXI.tga not saved

(this is the path to the local saved image location on the manager/server G4)

This error was seen on the server only (MacBook Pro) terminal window from where I launched the MacBook Pro server (./mxcl -server).

I stopped the job and the manager assembled the MXI. I do not know if this was simply the output of the one MXI on the manager machine or a combination of the MXI from both machines.

Posted: Fri Jul 07, 2006 6:46 pm
by mgroeteke
kirk, thanks a lot for doing those tests! i'll try soon again too with this new informations... what i'm still uncertain about is if it will be possible to stop the cooperative rendering at any time? will an assembled image be the result?

Posted: Fri Jul 07, 2006 6:48 pm
by kirkt
NOTE I have edited the above to reflect the fact that STEPS 5 and 6 are unnecessary.

Posted: Fri Jul 07, 2006 6:52 pm
by kirkt
mgroeteke wrote:kirk, thanks a lot for doing those tests! i'll try soon again too with this new informations... what i'm still uncertain about is if it will be possible to stop the cooperative rendering at any time? will an assembled image be the result?
You're welcome. I just stopped a job in mid render - select each node and hit the "Stop Job" button - the "State" changes from "Rendering" to "Waiting ..." waiting to assemble the image from the MXI. After you stop the job on all nodes, it appears that the MXIs are assembled into an image.

So, to answer your outstanding question, it appears that, YES, you can stop the job and the manager will take whatever has been rendered so far and assemble it into an image.

Now, can we continue rendering from where we left off....