No, not really, and you should find the time taken to be related to the number of places where the material has been assigned. When a material is removed, the plugin must traverse every entity (face, body, etc) in the model to find whether it uses the material in question, and if so, remove the assignment (and if real-time materials are enabled, update the appearance of the entity to reflect the change). It takes time to traverse the model, but it takes even more to unassign the material, especially in an assembly, when the plugin must ensure that the referenced part is loaded into memory. There are various SW API methods used to attach data, and to ensure that parts are loaded, and these can be quite slow to execute.
Not pertaining specifically to material deletion, but just more generally, it may be worth mentioning here that performance can often be improved substantially by avoiding per-face or per-feature assignments when possible; if you select a body with the Scene Manager > Object Properties tab open, you can see whether the "Enable Multi-Materials" toggle (the button at the right end of the toolbar) is enabled for the body; this is automatically enabled if/when you assign a material at the face or feature level (as is the global Multi-Materials option in Output > Export). This toggle is basically a promise you make to the plugin, which says that there are either no materials assigned at the sub-body level, or that you don't care if there are, and that the plugin can therefore take the shortcut of not checking for them, which can speed things up a lot for some models (mainly, it will speed up exporting to the renderer).