- Thu Feb 19, 2015 12:34 am
#385488
I thought it might be helpful to outline the overall functionality of the tp_network components and their parameters beyond the standard use case scenarios mentioned in the public docs. For official documentation, refer to the public docs at:
http://support.nextlimit.com/display/mx ... gy+Preview
Please note that the below info IS NOT OFFICIAL DOCUMENTATION FROM NEXT LIMIT and is merely my initial interpretation of how this all works and some questions where I just have no idea what I'm looking at. What I'm really trying to accomplish in this topic is to hopefully have Next Limit expand, critique, and correct the below info. Once refined, this info could be aggregated and condensed into 1 or more flow charts and may even help expand the public docs.
-------------------------------------------------------------
Basic Components of tp_network.exe (similar to old network tools):
Manager:
Since the interconnect between "tp_network.exe" on one host to another no longer limits itself to basic TCP/IP networking with plain text transactions and uses more common models like web servers, protocols (HTTP), and even other languages (Javascript & HTML), there are now apparently sub-components to the variants of tp_network.exe. To get a glimpse of some of these and to see additional parameters, we look at the help output of tp_network.exe:
Questions and Comments
---------------------------------------------------------------------------------
LOGHUB
---------------------------------------------------------------------------------
TREE
---------------------------------------------------------------------------------
WEB SERVER
---------------------------------------------------------------------------------
Somewhat vague parameters
---------------------------------------------------------------------------------
REP jobman socket?
---------------------------------------------------------------------------------
DUPLICATE DESCRIPTIONS
---------------------------------------------------------------------------------
This all seems like a good place to start. I know I'll have more questions once the above is clarified. Thanks Next Limit!
http://support.nextlimit.com/display/mx ... gy+Preview
Please note that the below info IS NOT OFFICIAL DOCUMENTATION FROM NEXT LIMIT and is merely my initial interpretation of how this all works and some questions where I just have no idea what I'm looking at. What I'm really trying to accomplish in this topic is to hopefully have Next Limit expand, critique, and correct the below info. Once refined, this info could be aggregated and condensed into 1 or more flow charts and may even help expand the public docs.
-------------------------------------------------------------
Basic Components of tp_network.exe (similar to old network tools):
Manager:
- Description - Handles the schedule of render tasks by delegating them to the nodes and reports status to monitors.
- tp_network.exe variant - tp_network.exe -manager
- Description - Server application that receives requests from the manager and does the actual render computation by launching Maxwell.
- tp_network.exe variant - tp_network.exe -node
- Description - User interface for scheduling tasks and also displays status information that it receives from the manager. As far as I know, the monitor never really communicates with any of the actual nodes. It only talks to the manager.
- tp_network.exe variant - tp_network.exe -monitor
Since the interconnect between "tp_network.exe" on one host to another no longer limits itself to basic TCP/IP networking with plain text transactions and uses more common models like web servers, protocols (HTTP), and even other languages (Javascript & HTML), there are now apparently sub-components to the variants of tp_network.exe. To get a glimpse of some of these and to see additional parameters, we look at the help output of tp_network.exe:
Code: Select all
---------------------------------------------------------------------------------C:\Program Files\Next Limit\Maxwell 3\tp_network>tp_network.exe -h
usage: mxcnetwork [-h] [-M MANAGER] [-v VERBOSITY] [-l LISTEN] [-j JOBMAN]
[-g LOGHUB] [-a JOBFILE] [-r] [-n] [-s SHARED] [-t TREE]
[-T TEMPDIR] [-p PARAMS] [-P] [-H] [-w] [-np]
[--address-webserver ADDRESS_WEBSERVER]
[--port-jobman-listen PORT_JOBMAN_LISTEN]
[--port-jobman-publish PORT_JOBMAN_PUBLISH]
[--port-params-listen PORT_PARAMS_LISTEN]
[--port-webserver PORT_WEBSERVER] [--port-tree PORT_TREE]
[--port-log-push PORT_LOG_PUSH]
[--port-log-web PORT_LOG_WEB]
mode
positional arguments:
mode Start jobman|render|web|client|params|tree|weblog|buff
ertofile|localshow 'version'or dump 'state'
optional arguments:
-h, --help show this help message and exit
-M MANAGER, --manager MANAGER
Address to listen on when manager mode is used, or to
connect to when render mode is used. If not specified,
auto-discovery is tried
-v VERBOSITY, --verbosity VERBOSITY
Verbosity (7 -> debug, 6 -> info, 5-> message,
4->warning)
-l LISTEN, --listen LISTEN
Address to listen on (server)/Address to listen to
-j JOBMAN, --jobman JOBMAN
Address of the jobmanager/dispatcherto connect to. If
not specified, auto-discovery is tried
-g LOGHUB, --loghub LOGHUB
Address of the log concentrator. If not present, log
only locally
-a JOBFILE, --add-job JOBFILE
Add job. Input specified as a file in protobuftext fmt
-r, --request-status Print current jobman status
-n, --dont-publish-address
Do not broadcast the job manager address in the
network (you will have to enter -j <ip> on render
nodes
-s SHARED, --shared-dir SHARED
Shared directory mount point
-t TREE, --tree TREE Address of the tree/params server (default, web server
IPaddress)
-T TEMPDIR, --tempdir TEMPDIR
Temporary dir(default
c:\users\zparri~1.c-s\appdata\local\temp )
-p PARAMS, --params PARAMS
ip,mxs,output
-P, --popen Use popen instead of subprocesses (not recommended)
-H, --threads Use threads instead of subprocesses (exp)
-w, --gui Run in gui (windowed) mode.
-np, --nopreview Do not send previews when in render mode
--address-webserver ADDRESS_WEBSERVER
Webserver IP address for js client (if different from
detectable IP address). This is for js only, web
server will still listen on all interfaces
--port-jobman-listen PORT_JOBMAN_LISTEN
Port for the REP jobman socket
--port-jobman-publish PORT_JOBMAN_PUBLISH
Port for the PUB jobman socket
--port-params-listen PORT_PARAMS_LISTEN
Port for the REP params socket
--port-webserver PORT_WEBSERVER
Port for the webserver to listen on (HTTP+Websocket)
--port-tree PORT_TREE
Port for the tree service to listen on (HTTP)
--port-log-push PORT_LOG_PUSH
Port for the tree service to listen on (HTTP)
--port-log-web PORT_LOG_WEB
Port for the tree service to listen on (HTTP)
Questions and Comments
---------------------------------------------------------------------------------
LOGHUB
Code: Select all
I know what log concatenation is, but I didn't realize for Maxwell it would be a separate, detachable component. How does this feature work and what is it used for? Does it somehow tie into the "File-> Save Logs" feature in the monitor?-g LOGHUB, --loghub LOGHUB
Address of the log concentrator. If not present, log
only locally
---------------------------------------------------------------------------------
TREE
Code: Select all
I'm really not sure what this component is and how to deliberately use it. -t TREE, --tree TREE Address of the tree/params server (default, web server
IPaddress)
---------------------------------------------------------------------------------
WEB SERVER
Code: Select all
This appears to be part of the manager. It's a miniature web server that temporarily listens on a custom port (other than port 80) and serves up HTTP requests that are really just simplified HTML wrappers for rather dynamic, compressed Javascript files. I did take a look at the Javascript that gets returned to the browser. Did you guys write all that completely from scratch or were you working from an existing platform like Node.js or Tornado?--address-webserver ADDRESS_WEBSERVER
Webserver IP address for js client (if different from
detectable IP address). This is for js only, web
server will still listen on all interfaces
---------------------------------------------------------------------------------
Somewhat vague parameters
Code: Select all
Are these parameters specific to the Maxwell Render engine or are they parameters for the Web Server? The reason I ask is that one of the selling points for Node.js is that it runs under a single thread. I thought that maybe these paramaters were for the Web Server to accommodate render farm layouts were you had several thousand nodes and the default Web Server modes couldn't handle all of the status and update requests. -P, --popen Use popen instead of subprocesses (not recommended)
-H, --threads Use threads instead of subprocesses (exp)
---------------------------------------------------------------------------------
REP jobman socket?
Code: Select all
My gut feeling is that there's a typo in the description. Maybe not, but I really don't know what "REP" stands for. I'm assuming this setting is actually the port that the manager uses to listen to the status responses of nodes.--port-jobman-listen PORT_JOBMAN_LISTEN
Port for the REP jobman socket
---------------------------------------------------------------------------------
DUPLICATE DESCRIPTIONS
Code: Select all
I grouped these lines together because their descriptions are all identical. I get the feeling that they were copy and pasted into the help text but actually need updated and are currently typos.--port-tree PORT_TREE
Port for the tree service to listen on (HTTP)
--port-log-push PORT_LOG_PUSH
Port for the tree service to listen on (HTTP)
--port-log-web PORT_LOG_WEB
Port for the tree service to listen on (HTTP)
---------------------------------------------------------------------------------
This all seems like a good place to start. I know I'll have more questions once the above is clarified. Thanks Next Limit!
Regards,
Zack Parrish
-
Maxwell - 4.2.0.3
Maxwell 4 | 3ds Max - 4.2.4
336 capable Maxwell threads!
-
Workstation:
Dual E5-2680v3, 64GB, Quadro K5200
48 threads (HT) @ 139.2GHz
-
Render Farm:
288 threads (HT) @ 835.2GHz
Zack Parrish
-
Maxwell - 4.2.0.3
Maxwell 4 | 3ds Max - 4.2.4
336 capable Maxwell threads!
-
Workstation:
Dual E5-2680v3, 64GB, Quadro K5200
48 threads (HT) @ 139.2GHz
-
Render Farm:
288 threads (HT) @ 835.2GHz