Dirty propagation issues on basic scenes

[pierre-ed]

We are experiencing stability issues with our scenes.
They only contain a moderate number of proxyshapes (say, less than a dozen) and yet we often end up with errors signalling unmet validation:

// Error: Failed verification: ' g_StageMap.isDirty() ' // 
// Error: Failed verification: ' fStageListeners.empty() ' //

Similar occurrences have been the subject of earlier discussions, especially this one which concludes on the seriousness of the issue and its root as a case of an illegal "pull evaluation during dirty propagation".

So we decided to investigate a bit to redesign our scenes and workflows and ended up a bit worried when trying to run cmds.dgdirty to check the state of our scenes' DG: Even on the simplest of examples, such validation errors occur.

Attempt 1

• We create an empty proxyshape node manually with node = cmds.createNode("mayaUsdProxyShape").
• We flag it as dirty with cmds.dgdirty(node).
• We click on the viewport (probably causing the proxyshape to attempt a redraw) and get the validation errors.

Attempt 2

• We create a proxyshape from a file from disk (a cube) via the menu entry Create->Universal Scene Description->Create stage from file
• We flag the proxyshape as dirty with cmds.dgdirty(proxyshape)
• We get the validation errors immediately.

We are quite puzzled by this behavior.
Is it expected? Are we missing something entirely? Are some proxyshape input plugs forbidden to get dirty?
We are looking forward to any kind of feedback because this lack of understanding hinders us from making our workflows less error-prone.

Thanks in advance.
Cheers,
-- Pierre

[williamkrick]

Hi Pierre,

I took a look at this one today and it turns out that using dgdirty on some plugs on the proxyShape is not safe, which surprised me.

What is happening is internally the dgdirty command has an optimization where it doesn't fully notify the dirtiness of plugs which are not connected. In these simple scenes the proxyShape's outStageDataAttr is not connected and so it's dirty notification is skipped: there is no call to setDependentsDirty with plug == outStageDataAttr. This leads to an unexpected circumstance where MayaUsdProxyStageInvalidateNotice has not been sent but the outStageDataAttr plug is dirty, which eventually leads to those assertions.

If instead of dirtying the full node you only dirty the input attributes you are expecting to change like so: dgdirty("sunglassesShape.time"); that will prevent you from the inconsistent dirty state and those messages. Other attributes that affect outStageData such as filePath will not trigger the error, again because of the same optimization. filePath gets dirtied but because it is not connected we don't do real dirty propagation and its affected plugs (including outStageData) are not dirtied.

I am curious about your original issue where you had multi-proxyShape workflows which would display the same assertions. Can you share a little detail or even better a reproducible workflow? What interaction with the scene triggered the warnings? Are the proxyShapes all displaying different stages or do some/all of them display the same stage?

If there is a more insidious issue with multiple proxy shapes I would like to identify it. If there is some other way to getting into this unexpected state that doesn't include the dgdirty command then fixing that issue would be a higher priority than the debugging workflow using dgdirty.

William

[pierre-ed]

Hi William,

Thank you for your quick answer and explanation.

To be perfectly honest with you, our workflows are currently using AL_usdmaya_ProxyShapes, not mayaUsdProxyShapes yet (mostly due to their ability to bring Maya references upon a variant switch).

Those validation errors thus occur with nodes that are about to disappear or maybe lacking support, and we wanted to assess to what extent switching to mayaUsdProxyShapes would solve everything (hence my very basics tests prior to any change).

It is not easy to provide you with a minimal scene that would guarantee such errors to occur: they mainly pop during the artists' work, when they are reparenting and renaming AL_usdmaya_ProxyShapes around.

They mostly happen in our scenes dedicated to sets, where we bring each individual set piece with a new AL_usdmaya_ProxyShape. Each one is specifically created with an individual session layer which becomes the edit target of the proxyshape's stage. So while these proxyshapes may load the same root layer on disk, they all have distinct sessions layers and do not share stages.

The operations those shapes then go through are basic in nature but possibly intensive in number: duplications, reparenting and renaming operations through maya.cmds. We also select variants by directly accessing their stage and using pxr.Usd API, editing their session layer in the process.

Also, I should mention that in our scenes, these proxyshapes have a custom locator as sibling, which helps us to store pipeline-relevant information at specific locations in the hierarchy. Hopefully, the presence of this extra shape next to the proxyshape does not interfere with it in any way.

Sorry for the long-winded answer, especially since our scenes are not using mayaUsdProxyShapes yet.

But as part of our questioning on whether we should move to mayaUsdProxyShapes now, do you feel like we are asking for operations they would forbid?

Thanks again.
-- Pierre

[womanyoyoyo-adsk]

Hi @pierre-ed,

Just wanted to let you know about a workflow we have on the mayaUsdProxyShape that mirrors your workflow involving multiple proxyShapes but not sharing stages. We introduced a 'Shareable' attribute on the mayaUsdProxyShape that uniquifies the stage. It is checked by default. When a user unchecks this attribute we add a new anonymous layer as the root layer and then sublayer in the file below it. Currently you can't save this new anonymous root layer to disk (as it would become shareable) but we do serialize it into the Maya scene file. For now, the only way to export this stage is procedurally.

I've made a quick demo for you showing this capability.

Documentation for this feature is found here: https://help.autodesk.com/view/MAYAUL/2023/ENU/?guid=GUID-5355AFE8-D39C-4950-A7BB-19407D59C626 under "Attributes for a Stage (proxyShape node)"

Hope that helps. Thanks,
Natalia

[womanyoyoyo-adsk]

When running cmds.duplicate on a mayaUsdProxyShape you will get a new, unique session layer in the duplicated mayaUsdProxyShape. When running cmds.duplicate on a mayaUsdProxyShap that is Unshareable which has edits in your unshareable layer, you will get the unshareable layer with those edits in the duplicated mayaUsdProxyShape.

[williamkrick]

I may have a couple of questions regarding scripting with this new attribute. Can the shareStage attribute be reliably edited with cmds.setAttr in the middle of a series of commands? Or are there specific commands to use, or some asynchronous process to wait for before resuming the script?

I tried this out and it seems to work as expected when changing shareStage from true to false. I created a scene with two proxyShapes sharing a stage. I ran a script which modified the stageStage attribute to be false for one of the proxyShapes, then I edited the USD in that proxy shape. The edit was display in only one of the proxyShapes, not both.

Going the other direction, from unshared to shared, is more tricky. If there is one proxyShape marked shareable with edits, and a second proxyShape which is marked unsharedable with edits, and then I mark that 2nd proxyShape shareable then what should happen to the edits?

[womanyoyoyo-adsk]

Going the other direction, from unshared to shared, is more tricky. If there is one proxyShape marked shareable with edits, and a second proxyShape which is marked unsharedable with edits, and then I mark that 2nd proxyShape shareable then what should happen to the edits?

I've made a demo of this scenario currently with mayaUsdProxyShape. The edits on the Unshareable layer are serialized into the Maya file. They reappear in the UI when the user clicks 'Unshareable' again.

[pierre-ed]

Hi Natalia, William.

Again, thank you both for your tests and explanations. It does help a lot.

We will try to move to the mayaUsdProxyShapes and test our different routines asap in order to get some real-life experiments and feedback. But before you go, I guess I'll have a final question though. :)

Up until now (and it may have been a mistake on our part, so I am eager for your insight), we created AL_usdmaya_ProxyShape nodes, sure, but in truth barely relied on the commands provided by the AL plugin.

Instead, we directly edited their respective stages through direct scripting with Pixar USD API... and avoided any asynchronicity issues by relying on a couple of callbacks on idle events (ugly, I know). But at the end of the day, our edits did end up pretty nicely in the layer targetted by the edit target of the different stages (in our case, their session layer), the Maya viewport reflected those changes, and all was relatively fine.

... Up until recently, when validation issues relative to dirty propagation and other viewport refreshing issues started to crawl into our scenes more and more often. So, I am trying to assess whether it is due to the fact that AnimalLogic's code is slowly getting deprecated or if our whole approach to stage edition was ill-advised to begin with.

So my question is : is directly editing proxyshapes' stages using Pixar USD API frowned upon? How prone is it to cause desynchronization between the USD data, maya_usd caching mechanisms, and Maya in general (its viewport render delegate or different editor panels)?

While our scripted edits are fairly tame (selecting variants, copying primspecs between session layers), I am getting worried our approach dooms our users to run into trouble using our code snippets now.

[williamkrick]

The general philosophy of the mayaUsdProxyShape is for it to faithfully display the current status of its UsdStage in the viewport through vp2RenderDelegate. With that in mind all of our code is listening to USD for changes to the stage rather than watching Maya tools. Using the Pixar USD API is a valid workflow and is supported by vp2RenderDelegate, and using the Pixar USD python API should not cause synchronization issues between the Usd Stage, the viewport or any Maya editors.

I created an example script that shows using the USD python API to directly modify a stage displayed in vp2RenderDelegate (based off code in https://github.com/Autodesk/maya-usd/blob/dev/test/testUtils/usdUtils.py).

Create the stage:

import mayaUsd.ufe
import mayaUsd.lib
import mayaUsd_createStageWithNewLayer

import ufe

from pxr import Usd, UsdGeom, Gf

usdSeparator = '/'

psPathStr = mayaUsd_createStageWithNewLayer.createStageWithNewLayer()
psPath = ufe.PathString.path(psPathStr)
ps = ufe.Hierarchy.createItem(psPath)
stage = mayaUsd.lib.GetPrim(psPathStr).GetStage()
aPrim = stage.DefinePrim('/A', 'Xform')
aXformable = UsdGeom.Xformable(aPrim)
aXlateOp = aXformable.AddTranslateOp()
aXlation = Gf.Vec3d(1, 2, 3)
aXlateOp.Set(aXlation)
aUsdUfePathStr = psPathStr + ',/A'
aUsdUfePath = ufe.PathString.path(aUsdUfePathStr)
aUsdItem = ufe.Hierarchy.createItem(aUsdUfePath)

bPrim = stage.DefinePrim('/A/B', 'Cube')
bXformable = UsdGeom.Xformable(bPrim)
bXlateOp = bXformable.AddTranslateOp()
bXlation = Gf.Vec3d(7, 8, 9)
bXlateOp.Set(bXlation)
bUsdUfePathStr = aUsdUfePathStr + '/B'
bUsdUfePath = ufe.PathString.path(bUsdUfePathStr)
bUsdItem = ufe.Hierarchy.createItem(bUsdUfePath)

Modify the stage:

bXlation = Gf.Vec3d(10, 8, 9)
bXlateOp.Set(bXlation)

If I select /A/B before running modify the stage I see Xform Op: Translate update correctly in the AE and the cube move in the viewport.