Page 1 of 1
Posted: Tue Feb 25, 2020 4:26 pm
I've reached a point where I have well over a thousand Maxwell materials on my system. These have been collected with little thought to how they would be used beyond any specific project and they simply reside in a folder on a networked drive. Inside that folder and (for most of them) subfolders that contain the actual MXM as well as the associated textures. Some of these are great, some not so much and some need to be remapped so that Maxwell knows where the textures are. In addition to the ones I've downloaded, I have an increasing amount of textures that I've created and I continue to make new ones in Substance.
My question is for folks who maintain a large texture library that they repeatedly go to when working on a project: How do you organize and catalog your textures for efficient use?
Right now I'm working on a limo interior and I need a leather material. I have several leathers that I downloaded from the Maxwell site along with maps I purchased to create hundreds more. Right now I go to the drive, and just keep opening folders until I find something I think I might like. I know Maxwell has the ability to maintain a material library but I'm not sure how to organize it and reference it when I keep the materials on a networked drive. I'm not even sure if this is the best method.
Any help would be appreciated.
Re: Material Organization
Posted: Wed Feb 26, 2020 2:15 pm
As for me, I created a directory called Maxwell_Mat_Library on a dedicated drive, and I simply created a lot of sub-folders in it, in conventional subjects. ("Metals", "Glasses", "Light_Emitters", "Woods", "Liquids and Foams", "Stone", "Plastics", "Concrete", "Cloth" "Landscapes", "Plant Parts" ...stuff like that). My Substances are arranged on this drive in exactly the same categories, as are my Maya Arnold materials. If I ever create a new category in one of these libraries, I stop and take the time to create it in the other material libraries as well. Plus, on this drive is a fourth Library of materials just called "Source_Materials" which are mostly the odd textures collected over the years. But this one also uses exactly the same general categories as the other three material Libraries, although I have many sub-specialty directories, such as for Japanese textiles, etc.)
The advantages of having a "Materials Libraries" drive, of course, are that you can back the whole thing up in one simple operation once a year or so, and you have room for the inevitable growth of the thing.
Like you, it took me a little while to realize I needed this discipline many years ago, and then a few hours to set it all up. About every year and a half, I go through these libraries to clean them up - the same time as I am archiving old projects and cleaning up my drives. The systematic material libraries have been worth every minute I spent on the initial organization in all the time since then. By the way, my Windows "View" settings are turned on to "Large Icons"for each of these directories and sub-directories, so I can see what the materials looks like as soon as I open the directory. My Maxwell Studio preferences are pathed to search that main Maxwell_Mat_Library, of course. (Just like Substance Painter and Substance Designer are pathed to find my main Substances library.)
When I start a new Maya project, I have a little script that automatically creates some additional directories in each Project Folder Set. (Or you easily can adjust the Maya Preferences settings to replace some of the defaults you'll never use.) These automatic additional Maya Project sub-directories are "Maxwell_Scenes", "Maxwell_Mats", "MAX" (for probable Autodesk MAX conversions), "Arnold_Mats", "FBX_s" and "OBJ's". (A "Substance" sub-folder now is getting created automatically with current versions of Maya.)
If I can anticipate which mats I'll need, I then copy over the relevant Maxwell materials out of my main library to the Maxwell_Mats directory in the Maya Project Folder before starting the modelling process. (Same for any Arnold materials.) This is useful because there is often a need to tweak a material for a specific project, while you want to preserve the original in the main Maxwell Mat Library. If I can't anticipate which materials I'll need at the beginning, as I discover I need one, I still copy it from the main library to the individual Maya Project File set . And I am religious about tweaking only the Maxwell material within the Project file set. This kind of preserves the record of what I did for each individual project.
Having the Maxwell_Mats sub-folder in each Maya project file set is important for two reasons.
First, it leaves my RAL color libraries and my Federal standard architecture materials libraries intact in the main Maxwell_Mats_Library. For my main Maxwell Materials Library, I also have a large, very systematically color tinted set of liquids and another library of light emitters that never get messed with. Only if the tweaked Maxwell Material for an individual project is truely wonderful and something I certainly will draw on in the future, do I promote it up the the main Maxwell_Mats_Library. (But happens all the time with things like green weathered bronzes, specialty metals, glasses and lights.) Otherwise, I leave it in the individual Project file set.
The second reason is that you probably have to travel a thousand times between Studio and your modelling tool over the course of a project development. You only want to create each material one time if you can humanly do so. So, go to Maxwell early on in your modelling process or scene development process, create that material as early as you can (or copy it down from your main Mats_Library), and then immediately preserve it in your project directory.
The discipline and organization saves thousands of hours in actual project work.
Re: Material Organization
Posted: Wed Feb 26, 2020 3:08 pm
Thank you for this fantastic reply. This is somewhat similar to what I recently started doing and I think I will adopt this in full. I also copy models from a main model library over to the assets folder in Maya, though I'm not sure if this makes sense.
I want to reorganize all my materials the way you suggest but if I do it now, I know I'll break quite a few scenes that I'm not finished with yet. I'm also toying around with a FileMaker database that would contain all my materials (both Maxwell and Renderman as well as Arnold Materials, Substance Materials, etc.) and all texture maps. This would allow for being able to keyword search for what you want and then be able to export the material directly to a materials folder in a project (as you describe). Eventually I would like to do the same for my collection of 3D assets as I have by now accumulated quite a few of them.
If I may ask, why do you prefer texturing and rendering in Studio? Since you mentioned it in another post, I downloaded the demo and have been working with it (thank you for the camera reply, btw). I'm not sure I understand the benefit of working in Maya and then in Studio as opposed to just doing everything in Maya. If you could explain your process there, and why you do it, that would be amazingly helpful.
Thank you so much for your time, it is truly appreciated!
Re: Material Organization
Posted: Wed Feb 26, 2020 7:12 pm
Apologies for making just a short reply here - I seem to be coming down with a bad cold. Am looking for a bed somewhere.......
A comment on your Filemaker idea: I'm an old programmer and Microsoft Access database maker, so I'd sort of had the same idea as this one. Did quickly make such a database - with images, and mostly so that I could comment my materials and keep notes on them. But, honestly, I rarely used it, and ditched it after it sat around for a year mostly unused. In my case, I think it took more effort to maintain it than it was worth. I see that you are a MAC user, and I'm pretty ignorant of the Apple Operating systems, so this may be more useful to you than I think. Especially, the ability to comment out a material. In Windows, we can set the directory displays so that they are alphabetical if we wish, and to automatically display any icons associated with each file. If you have "Updated" a Maxwell material, it does automatically create an "icon" which really is the "Preview Display" of that material, and is at a decent resolution. So, whenever I open a directory - maybe of the "Metals", there are all my Maxwell materials on full color display with a small title underneath. Sometimes, I have to do a little bit of work to keep the associated textures properly labeled, but that is minor. And this has been entirely sufficient to my needs.
Once we get a Maxwell version that makes direct use of Substance materials, I'm probably going to have to rethink this, but it may be that it still will be easy, given than my Substance and my Maxwell directory structures are identical down to at least two levels in their hierarchies. Anyhow, your Filemaker idea might prove best when this Substance connection comes into being.
As to my preferences for Studio over the plugin, somehow, the Studio interface just seems to give me a more direct and intuitive platform or interface to what I'm working on. The material editor is there right in front of me, the Asset Library is right there in front, the Console Dialogue is there , the Camera controls are front and center, and its easy and direct to access any of the other features. Neither Maya nor that Autodesk Desktop App are stealing any CPU cycles, so I know all my RAM and the cache are focused on the renderer and its operations. It is admittedly a pain in the arse to first have to export any *.fbx files out so that they can be loaded into Maxwell, but once most of materials are made, it really is not too bad to go back and forth. For rough work, I do keep Maya going - so both Maya and Maxwell Studio are open and using the CPU at the same time. But, when I start to zero in on the rendering processes, I close Maya and everything else so that Studio is able to use all my computer resources.
Maybe this works well, because I build my own computers, and so always have at least two of them. One is dedicated to Maya, MAX, Realflow and Maxwell Render. (Plus nBalancer - a decent polygon reducer tool.) The other one, the Utility machine, has the web browser, Microsoft Office products, my programming tools, and all the tons of other tools and utilities one needs. So, I think I might be jealously guarding my CPU cycles for whatever part of the pipeline I'm working on at the time, and that is why Studio works better for me.
You mentioned making use of the "assets" sub-directory in the Maya Project Files set. Personally, I preserve this sub-directory for storing and using modelling assets I acquired elsewhere. Maybe, I made a chair earlier for another project, and I want to use that in the present project. Maybe my buddy Clark made a special light and he's given it to me for the present project. You know, "foreign" stuff. I think this is what the Maya designers intended, and some of the utilities I use in Maya seem to expect this kind of use of the "assets" sub-directory also. But, of course, you can do anything you want with your own Maya setups.
Whoops, go to go....
Re: Material Organization
Posted: Thu Feb 27, 2020 9:08 pm
Very good question.
I am also interested to know how other users manage there files...I mean textures (from the substance tools or other ressources)
As .sbsar is not compatible rigth now, we are talking about bitmap textures right?
Thanks to have asking, I was to shy ...
For your question related to the usage of MR Studio prefered to the plugin (I also use Maya on Windows)
I tend to use Studio depending on the deadline...if i have the time and still motivated by the project (sometimes I get tired with architectes)
I export at the very end of the project for final render, because the plug-in for maya seems to export a very clean and optimised scene file.
Writen just for maxwell and the calculation process.
Between us, it's been six months since I have not used Studio, never time... I have to stay inside Maya to modifie the scene as quick as possible.
But it 's a bad habit more and more due to tight dead lines...Like this you lose your quick hability inside studio, even if it'is very intuitive.
I encourage you to take a deeper look at it, very quick and easy learning curve.
As Forester say, everything is in front of you on the screens.
Hope your bad cold is getting better!
Thanks to share your managing workflow, I like your point of view as an old programmer, very interesting and convincing!
Your system and your rigor seems to be very clean. It is exactely a good practice for managing suche a big volume of data of different kind.
I mean bitmap, procedural, etc...
I tend to use the same approch as you in the local machine.
Therefore, I have a question:
How do you backup everything periodicaly?
I mean bitmap, procedurals, finished main materials, Maya projects directories with duplicated tweaked textures/materials and other data that to have to belong inside the maya project hierachy?
Do you have a kind of server to archive periodicaly? Do you use a kind of cloud system that syncronize always your directories?
It would be very pleased if you could explain me a bit more as I am looking for a home system that doesn't steal any CPU ressources while I'am working.
Until now, I do it manually time to time but it's boring, and a bit risky if one of my local drive suddently die (never happened since almost 20 years)
Other unrelated question:
are you a bonsai trees lover? Because of your avatar
Thanks boths, cheers!
Re: Material Organization
Posted: Fri Feb 28, 2020 11:17 am
Keeping materials and textures organised was a big consideration when I first started using Maxwell- from V3. I already had a largish texture library thanks to being subbed to what is now cgtextures.com. This is arranged into the usual texture groups such as bricks, concrete, wood etc, each with sub folders for manufacturers specific maps where applicable. A lot of my generic textures are not in sub-folders.
I didn't want to split my textures up to different locations when transitioning to Maxwell, so they are all located in a separate folder on a dedicated internal hard drive/partition. I have another library folder dedicated for mxm files, which is categorised nearly the same as my texture library, but still logical and easy to find stuff. When creating a new mxm and prompted to copy textures to the mxm folder, I decline.
I now also have separate folders for Substance materials ie. sbar and sbs, but the actual texture maps are saved in my textures directory. I did start to maintain a separate texture folder for UE4 textures, ones that were reduced down and scaled to a factor of two, but what with using Substance and the textures at Poliigon these days, I have stopped doing this.
My main issue with finding textures is Windows 10 bloody awful search (dis)functionality. Browsing through a folder with 100 or more textures is great to quickly see what may work in a particular scene, but when a specific texture is needed, the search doesn't usually work and I have to scroll through everything anyway to find the one I am looking for. However, I have recently discovered Connecter, by Design Connected. It is quite a good browser, similar to Adobe Bridge in some ways, but can view thumbnails for 3D models in lots of different formats, as well of course texture files and HDRis etc.
Re: Material Organization
Posted: Wed Mar 04, 2020 6:45 pm
Thank you, buffalo deal. And thank you for the complements.
About backups, I use a solid state drive that has a USB 3 connector where the SATA connector would normally go. (Here is an example of this kind of connector. StarTech.com USB 3.1 to 2.5" SATA Hard Drive Adapter - USB 3.1 Gen 2 10Gbps with UASP External HDD/SSD Storage Converter (USB312SAT3CB)). A solid state drive is relatively inexpensive and is extremely reliable. I do not compress the backups - just run a script that copies all the directories and files for any directory that is "new" and for any files that have changed since the last operation. If the backups are not compressed, they are immediately available for access or copying, in case that is needed.
For some client machines, when I build them, I often add a third or fourth SATA-cabled solid state drive inside their computer box, specifically intended for periodic backups. Many people do not remember to make backups, and it is easy to write a Windows script that will automatically make backups for people on a set schedule, in the background. It protects people who cannot remember to make regular backups. But, if you do this, you almost always need to use a compressed backup format. I don't like to use the Windows standard compressed backup, because in my experience, it can fail. I would rather make a direct copy for my own projects, because that is more reliable, and the SSD storage is cheap enough that I am willing to use the space and the time to make this kind of backup.
Yes, I have admired bonsai for many years. I really am a professional forest ecologist - have grown many bonsai myself. Not very good at it, however.
Re: Material Organization
Posted: Wed Mar 04, 2020 6:50 pm
Thank you for your reply, CDRDA. I had forgotten about the Design Connected "Connector." It is a good tool and is recommended. It works well with MAX and C4D, but not quite as well with Maya. (It will read a Maya file (both *.ma and *.mb), but requires quite a lot of set up to work with a Maya project. (In my humble opinion.)) It does a very good job of displaying assets in *.obj file format if they have a *.mtl file with them, especially at the 400 x 400 pixel size, and it does a great job of displaying the scale of your assets.
Windows 10 Extra Large icon settings does almost the same thing, but the "Connector" has the advantage of being able to generate a *.pdf file whenever one is needed. Of course, it is a proprietary software, designed to connect and allow you to scroll through Design Connected's holdings. And it leaves a lot of personal identification stuff in your Windows Registry if you uninstall it. Probably works best for MAX and C4D users.
For Maya users, I really recommend the Ninja Dojo tool set (https://www.bk3d.com/Ninja_Dojo/
). Cost is $60.00 USD. A little expensive, but you only pay the cost one time, and all upgrades are free after that.
One of the many features of this set of Maya tools is that you can easily set it to display all your assets in any given project file set, although it will only do so in Maya Software or VRay format and it displays them as 200x200 pixel size. Very primitive, compared to the Connector. But, it will immediately show you your files that are in *.ma, *.mb, *.fbx and *.obj format, while you are at work within Maya. And, you can easily make a visual search of any other project directory for other files, if you want to locate some other Maya file and bring it into your current project.
By the way, the Ninja Dojo has a fantastic set of tools. I cannot praise these enough: I use them every day. They save me hours and hours of time. There are the usual tools for selections, edits, mesh creations, spline and lots of things you already can do in Maya, but these are optimized to save a few clicks, or their most used settings are easily accessible. Then there are tools for things that take a more work if done in Maya, such as randomizations, simple things like renaming a hundred objects, spline to poly tubes or flat faces, scattering, fractures, gang-export and import of multiple individual objects in *.obj and *.fbx format, etc. And, you can easily create a "chain" - record a set of a repetitive actions for re-use without having to stop to write a MEL or PY script. I use this one all the time. And speaking of scripts, the Dojo will let you quickly create a catalogue of and access all your MEL scripts, just like all your assets, so that they are handy for any project, without having to clutter up your working shelves. It does a ton of different things for which you've probably already downloaded a lot of little individual MEL or PY scripts, but the tools are neatly organized in a way that makes sense, kind of like Maya's Mesh Modeling Toolkit. Anyhow, its well worth the price.
Re: Material Organization
Posted: Mon Mar 09, 2020 11:11 am
Thank you Forester,
Very helpfull, gonna study your way to proceed