Entity's transient attribute's value is lost - was OK in previous Jmix versions

[chrisbeaham]

Jmix version: 1.5.1
Jmix Studio plugin version: 1.5.2-231
IntelliJ IDEA 2023.1 (Community Edition)
Build #IC-231.8109.175, built on March 28, 2023
Runtime version: 17.0.6+10-b829.5 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
GC: G1 Young Generation, G1 Old Generation
Kotlin: 231-1.8.20-IJ8109.175
Java 17.0.4 2022-07-19 LTS
Java™ SE Runtime Environment (build 17.0.4+11-LTS-179)
Java HotSpot™ 64-Bit Server VM (build 17.0.4+11-LTS-179, mixed mode, sharing)
Operating System: macOS 13.2.1 (22D68)
File System: Case-Sensitive Journaled HFS+ (APFS)
Datebase: PostgreSQL 13

Hello Everyone

For your information, I have a problem with a transient Boolean attribute. My code was working in CUBA and for several versions of Jmix after my migration; I found the problem last week before upgrading to version 1.5.1.

The idea is:

• Entity A contains two many-to-many relationships, one to instances of Entity B and one to Entity C.
• In my Entity A edit screen I have a separate table for each of the Entity B and Entity C instances.
• When the user creates a new Entity B instance, he has the option in the Entity B edit screen to automatically create a new corresponding Entity C instance.
• This option is represented by a transient Boolean attribute in Entity B, e.g. createEntityCInstance.

The process is:

• In my Entity A edit screen the user chooses “Add” from the Entity B table.

    @Subscribe("entityBTable.add")
    public void onEntityBTableAdd(Action.ActionPerformedEvent event) {

        screenBuilders.lookup(entityB.class, this)
                .withOpenMode(OpenMode.DIALOG)
                .withSelectHandler(entityB ->
                        entityB.stream()
                                .map(this::createEntityB)
                                .forEach(this::addEntityB)
                )
                .build()
                .show();
    }

• An Entity B browser screen is presented to the user as a dialog.
• If the user does not find the Entity B instance that they want in the browser list, then they can choose “Create” in the Entity B browser screen.
• The newly created Entity B instance, with the transient “createEntityCInstance” attribute set to “false”, …

    @JmixProperty
    @Transient
    protected Boolean createEntityCInstance = false;

is passed on to the Entity B edit screen …

    @Subscribe("entityBsable.create")
    public void onEntityBTableCreate(Action.ActionPerformedEvent event) {

        entityB entityB = dataContext.create(entityB.class);

        screenBuilders.editor(entityBsTable)
                .newEntity(entityB)
                .withOpenMode(OpenMode.DIALOG)
                .withScreenClass(entityBEdit.class)
                .withAfterCloseListener(afterScreenCloseEvent -> {
                    if (afterScreenCloseEvent.closedWith(StandardOutcome.COMMIT)) {
OK here ->              entityB newEntityB = afterScreenCloseEvent.getSource().getEditedEntity();
after Close             entityBTable.setSelected(newEntityB);
                    }
                })
                .build()
                .show();
    }

• The user must choose some other Entity B instance attributes and they can select a checkbox to set the transient “createEntityCInstance” attribute to “true” to optionally create a new corresponding Entity C instance.
• When the user presses OK in the Entity B edit screen, the new Entity B is saved (commit) and the user is returned to the Entity B browser screen and the new Entity B instance is automatically selected; see the withAfterCloseListener() code above.
• At this point in time everything is correct; I can see in the debugger that my new Entity B transient “createEntityCInstance” attribute is still set to “true” in the Entity B browser screen; see “OK here → after Close” above.
• When I press “Select”, which closes my Entity B browser screen and returns the user to the Entity A edit screen, I return the new Entity B instance in the withSelectHandler() code but at this point the transient “createEntityCInstance” attribute is not “true” anymore; therefore, a new Entity C instance is not created as was requested; see “NOK here → after Select” below.

This is the same code as listed above in the first section …

    @Subscribe("entityBsTable.add")
    public void onentityBsTableAdd(Action.ActionPerformedEvent event) {

        screenBuilders.lookup(entityB.class, this)
                .withOpenMode(OpenMode.DIALOG)
                .withSelectHandler(entityB ->
NOK here ->             entityB.stream()
after Select                    .map(this::createEntityB)
                                .forEach(this::addEntityB)
                )
                .build()
                .show();
    }

Originally, the transient “createEntityCInstance” attribute was not selected in my fetchPlans.xml, so I added it everywhere yesterday but the result is the same.

Can you please help me resolve this problem.

Thanks in advance.

Best regards
Chris

[taimanov (Dmitry Taimanov)]

Hello Chris,

Sorry for the late reply.

Thank you for the detailed description of the problem! Unfortunately, it is still too many details in entity and screen implementation that is unclear and can affect issue reproducibility.

Could you please provide a minimal example project to reproduce the problem?

As far as I understand for now, you are passing a screen behavior flag using transient attribute of a newly created entity. This entity is returned to the main edit screen, added to the table and participates in other screen lifecycle events.

It looks unsafe because the entity can be reloaded and a transient attribute will be lost (e.g. during table refresh).
Maybe it is better to use some custom outcome for the screen? or get property from entityBEdit screen directly in afterCloseListener?

if(afterScreenCloseEvent.getSource().isCreateEntityCInstance()){
   // open entity C creation screen or remember in some variable that C for current B should be created too?
}

Regards,
Dmirty.

[chrisbeaham]

@taimanov

Hello Dmitry

Thank you for your feedback and no apologies necessary. This should not be unsafe because in my application other users only have read access to these entities and only after the main Entity A instance is explicitly published using another Entity A attribute, which is referenced when other users are searching.

Furthermore, if the transient attribute is set to true in the Entity B edit screen during Entity B creation, and then returned to the Entity B browse screen and the user decides to refresh the Entity B browse list or choose a different existing Entity B instance from the Entity B browse list to return to the Entity A edit screen, then nothing is actually lost because my users have the option to manually create an Entity C instance at any time in the future. My Entity B and C are closely related, so I am just providing the user with the opportunity to directly create an Entity C instance during the Entity B creation to save the user a few mouse clicks.

I have built an example project and it works as expected; please find it attached. My example has an Entity C but it is not used at all. Also note, that the createEntityCInstance transient attribute can only be set during Entity B creation; if edit is selected for an existing Entity B instance, then I do not display the createEntityCInstance option in the Entity B edit screen.

losttransient.zip (96.7 KB)

Please have a look at my example and I will continue to try and analyze what might be causing this. Thanks in advance for your further support.

Best regards
Chris

[taimanov (Dmitry Taimanov)]

Hello Chris,

I have looked at your project: it works, and the attribute is preserved after the creation: the message “Create Entity C Instance - SELECTED” appears.

Is the issue cannot be reproduced in this example? Until I have an reproducible example to investigate I can suggest only next idea:
Look at the next things during the debugging of the original project:

1. Is the entity with the lost attribute the same java object as the entity where the attribute is still true?
   1.1 If it is the same java object, put the breakpoint inside setCreateEntityCInstance and look what method changes the value.
   1.2 If it is another java object, put the break inside the constructor (add default one) and investigate when the new entity with lost attribute value is created. Stacktrace will show when and why the entity is cloned.

Regards,
Dmitry

[chrisbeaham]

@taimanov

Hello Dmitry

Thank you for your assistance and the suggestions. I have found the problem and it is a timing problem or race condition but I do not know how it is caused. First let me answer your questions…

I have confirmed that all of my screen objects use the same Entity B java object, for example…

Entity B edit screen: b17df107-d1f3-bbf0-d43e-ba34cd7b633d [new]
Entity B browse screen: b17df107-d1f3-bbf0-d43e-ba34cd7b633d [detached]
Entity A edit screen: b17df107-d1f3-bbf0-d43e-ba34cd7b633d [detached]

I put a breakpoint inside my setCreateEntityCInstance method and it is called to set the attribute to true but it is not called after that to set the attribute to false. Therefore, the attribute is being set to false by some side effect somewhere.

I copied some more of my project code into my example project, to try and align them closer, however, this did not affect my example project; it still works correctly. Here is my new version.

losttransient.zip (97.2 KB)

After checking all of this, I decided to put a breakpoint inside my getCreateEntityCInstance method and after doing that, my project works correctly as it did before; the Entity B createEntityCInstance attribute is still set to true when returning the new Entity B instance to my Entity A edit screen. This means that there is a timing issue somewhere.

I have reproduced this behavior 7 times (every time). I originally had 12 breakpoints and I disabled all of them except the getCreateEntityCInstance method breakpoint and everything works when this breakpoint is still active. If I deactivate it too, then createEntityCInstance is set to false somewhere.

What do you recommend that I try/analyse now?

Thanks in advance for your feedback.

Best regards
Chris

[taimanov (Dmitry Taimanov)]

Hello Chris,

As far as I can see from the next lines, Entity B is the same business object with the same Id but it is not guaranteed that it is the same java-object:

@chrisbeaham

Entity B edit screen: b17df107-d1f3-bbf0-d43e-ba34cd7b633d [new]
Entity B browse screen: b17df107-d1f3-bbf0-d43e-ba34cd7b633d [detached]
Entity A edit screen: b17df107-d1f3-bbf0-d43e-ba34cd7b633d [detached]

Just in case of this thing is not checked yet: each object listed above can be a different java-object, i.e. different clone of the same business object. In order to check this case, please have a look at the object in the debugger and make sure that the underlined numbers are the same for entity at each moment of time:

Otherwise, this entity has been cloned/reloaded during the screen lifecycle. In order to catch this moment it makes sense to put a breakpoint into the constructor and investigate when cloning/reloading occurs.

Transient attribute value will be lost during entity reloading, so either this value has to be processed before this moment or has to be passed using another way (e.g. through window closing event).

Best regards,
Dmitry

[chrisbeaham]

@taimanov

Hello Dmitry

Thank you for your feedback and the further suggestions. It was my misunderstanding that I mixed up the java object ID with the business object ID; I was not aware of this java object ID; I never had to monitor it before. You mention an object’s clone several times, however, my understanding of a clone is that it is identical to the original object. Therefore, I do not see why a clone of any object would cause an object’s attribute value to change. The cloning behavior must be completely independent of the type of attribute. I assume that when the Entity B business object is loaded from the database and its transient’s value is added to the resulting java object, every following clone operation to the java object would have no effect on the attributes’ values. The framework must ensure this. Or am I missing something here? A reload is something completely different.

I tried to add a constructor to my Entity B yesterday but the compiler did not like it, which I could not figure out (I had never done this before), and since I did not have a lot of time, I did the following instead…

1. I confirmed that my business object is represented by 3 separate java objects:

• 1st after the object creation and used in the Entity B edit screen
• 2nd after the Entity B edit screen closes and returns to the Entity B browse screen (clone #1)
• 3rd after the Entity B browse screen closes and returns to my Entity A edit screen (clone #2)

2. I manually changed 3 attribute values of my Entity B business object directly in the database after arriving at a breakpoint in my Entity B browse screen controller when returning from the Entity B edit screen.

3. I then monitored whether these changed DB values were reloaded when the Entity B was selected in the Entity B browse screen table and returned to the Entity A edit screen.

• The new database attribute values were not loaded into my Entity A edit screen’s table that contains Entity B. All of the attribute values were taken from the Entity B object that was returned from the Entity B browse screen.

4. When I closed and reopened my Entity A edit screen, the values that I manually changed in the DB where shown as expected.

5. I repeated steps 2 – 4 five times and the results were always the same: there was no reload of Entity B from the database.

6. I also monitored my Entity B PostConstruct method and it was only called once during the first Entity B creation.

After that, I simply added a breakpoint to my getCreateEntityCInstance method again (like last time) and then the attribute’s value was “true”, as expected. Why does it work when the breakpoint is set?

In the meantime, I see how I can implement this through a window closing event but I would like to find the root cause of this problem because the breakpoint side effect may also be affecting other functionality.

What else do you recommend?

Best regards

Chris

[taimanov (Dmitry Taimanov)]

Hello Chris,

I assume that when the Entity B business object is loaded from the database and its transient’s value is added to the resulting java object, every following clone operation to the java object would have no effect on the attributes’ values. The framework must ensure this. Or am I missing something here?

Yes, in general cases “cloning” (or copying) operation should preserve transient attribute values. It is a thing to be aware of just in case.
But it is also more complicated operations during entity workflow. E.g. merging of changes between two clones of the same entity.

What if we have one clone loaded in the parent screen (e.g. EntityA editor) and another clone returned from nested screens (e.g. EntityB browser, opened from the EntityA editor)? We have to merge changes made in these entities.

So, how do we know which attribute value to prefer during the merge of band b' entities? Which ones have been changed by the user and which ones are just loaded from db? How to prevent the loss of user changes?

This problem is solved for persistent attributes using the system objects such as ObjectChangeSet deep inside the entity and/or user session. But changes in transient attributes are not tracked and may be not transferred correctly.

Actually, it is a very imprecise and shallow example of what can happen. Maybe it is something else in your case. Unfortunately, I cannot say what exactly happens without a reproducible example.

But the main thing to remember: transient attributes is not a reliable way to transfer data between screens.

I tried to add a constructor to my Entity B yesterday but the compiler did not like it

The constructor for EntityB in the last project you provided:

//...
public class EntityB {
    @JmixGeneratedValue
    @Column(name = "ID", nullable = false)
    @Id
    private UUID id;

    @Composition
    @OneToMany(mappedBy = "entityB")
    @OnDelete(DeletePolicy.CASCADE)
    private List<EntityBMapping> entityBMapping;

    @InstanceName
    @Column(name = "NAME", nullable = false)
    @NotNull
    private String name;

    @JmixProperty
    @Transient
    private Boolean createEntityCInstance = false;

    public EntityB() {
        System.out.println("EntityB created.");//TODO: do not forget to remove after debug
    }
    //...

Why does it work when the breakpoint is set?

Apart from some kind of race condition in your app, the breakpoint may cause the computation of entity fields to show them in the debugger. This computation will trigger lazy loading of reference collection fields: a separate request to the database, loading of nested entities, adding them to persistence context, considering them during merge and saving of entities, etc.

In order to avoid this it is better to disable the “collection” view of fields in the debugger. E.g.like this:

So, in order to exclude lazy loading side effect of debugging, your fields should be displayed in debugger in such way:

If you see the size of collection - then it has been loaded either normally, or by the lazy loading during the debug.

===========

So, in general, If you want to know what exactly happens, I would recommend putting a breakpoint inside the constructor and inside the property setters and carefully watching what happens with the transient attribute (with disabled collection display as on screens above).

But honestly, I can come up with only two reliable ways to resolve this situation: either make the transient field persistent or transfer values through window closing event.

Best regards,
Dmitry

[chrisbeaham]

@taimanov

Hello Dmitry

Thank you very much for the detailed description and the further suggestions. I apologize for my late reply but it was unavoidable.

In the meantime I performed multiple tests on my main computer (iMac) and also on a laptop and the behavior is not deterministic. For example, I added just the constructor (below) as you suggested, and without any breakpoints set, the addition of the constructor alone changed the behavior; everything began working again on my iMac. I then did the same thing on my laptop and the addition of the constructor had no effect. A week later I repeated the tests on my iMac and the behavior was incorrect there too.

public EntityB() {
        System.out.println("EntityB created."); //TODO: do not forget to remove after debug
}

I also disabled the “collection” view of fields in the debugger as you suggested but this had no effect; it still worked as expected with the getter method breakpoint active.

This information from you seems to be the most relevant to my situation …

“This problem is solved for persistent attributes using the system objects such as ObjectChangeSet deep inside the entity and/or user session. But changes in transient attributes are not tracked and may be not transferred correctly.”

… and I will therefore stop my efforts to find the root cause and change my implementation; that is time better spent.

Thanks again for your support.

Best regards
Chris