Liquibase improvements

I have read this thread JMIX with Liquibase - #4 by krivopustov
and I agree that such a checkbox would be very useful, and I have some ideas to propose further so that studio users can get more control over Liquibase.

  1. ability to invoke Changelog Preview from the Jmix Data Store menu - now it’s appearing only after one starts Run Jmix Application
    This is a very useful screen that already exists - perhaps it can be invoked with “Save and run” and “Discard and run” buttons set to disabled and not visible
    image
  • a sort of “simulated run”
  1. ability to right-click an entity, or a table in such screen, and have few simple preconditions to toggle from Liquibase changelog , such as:

create table if not exists

<changeSet id="create-table-if-not-exists" author="mladen">
    <preConditions onFail="MARK_RAN">
        <not>
            <tableExists tableName="my_table"/>
        </not>
    </preConditions>

or add a column if not exist for fine-grained control

<changeSet id="add-column-if-not-exists" author="mladen">
    <preConditions onFail="MARK_RAN">
        <not>
            <columnExists tableName="my_table" columnName="email"/>
        </not>
    </preConditions>

and if exists leave it alone , this is important for the platform and addon entity changelogs
As well as related common useful preconditions.

  1. Granular entity attribute control - example I have a Map add-on, I change the geo polygon default size from its small default to say 40k to accommodate the size, if this “modifyDataType” is not ignored it can truncate the data, so it would be good if we can right-click column name/attribute somewhere and set it to not be touched

  2. Something cool but not really important if we add some manual precondition like this example

        <preConditions onFail="MARK_RAN">
            <not>
                <and>
                    <sqlCheck expectedResult="1">
                        SELECT COUNT(*) FROM public.PCL_USER WHERE ID = 'xx-x-x-xxx-x'
                    </sqlCheck>
                    <sqlCheck expectedResult="1">
                        SELECT COUNT(*) FROM public.PCL_USER WHERE USERNAME = 'admin'
                    </sqlCheck>
                </and>
            </not>
        </preConditions>

→ have Studio mark somehow the changelog that was manually edited
Also, preconditions are cool if you write an application that uses several database types

<changeSet id="mysql-specific-change" author="mladen">
    <preConditions onFail="MARK_RAN">
        <dbms type="mysql"/>
    </preConditions>
    <createTable tableName="my_table">
        <column name="id" type="bigint" autoIncrement="true">
            <constraints primaryKey="true"/>
        </column>
        <column name="name" type="varchar(255)"/>
    </createTable>
</changeSet>
  1. Does this deserve, if gone far, a central Liquibase management screen where all this can be invoked, changelogs edited, entities right-clicked, simulations run, Ignored section managed and viewed?

Kind regards,
Mladen

Hi Mladen,

Some of your suggestions are already implemented, others I would like to discuss.

  1. The “Generate Liquibase Changelog” action in the data store context menu does exactly what you need – just creates and saves a new changelog without starting the application. See Data Stores :: Jmix Documentation. And you can change the generated changelog file manually before executing it.

  2. Do you mean when editing the generated changelog? Just a quick way of adding these preconditions instead of writing them manually?

  3. The framework provides the @DdlGeneration annotation that you can use for granular control over changelog generation for an entity. For example, you can add your column to the unmapped columns list. And Studio provides a UI for managing this annotation, see DDL Generation Settings in the entity designer.

  4. Not sure Studio can distinguish what was manually changed. Maybe just change the file name when modifying the changelog manually? You can easily rename the file while it’s not yet saved. Afterwards, you can do it only in the Project tool window. We deliberately don’t let users rename changelogs in the Jmix tool window because it’s dangerous.

  5. Interesting idea. Looks like currently the existing functions are hard to find.

Regards,
Konstantin

Hello,

With time, I got Liquibase to do what I want, but it was a steep learning curve and sometimes I don’t feel enough in control. Maybe it’s subjective, but I can also see a lot of Liquibase problems on the forum and Slack. Maybe some blog about its use, or a cheatsheet is needed too. Definitely there is some low-hanging fruit to improve Jmix studio.

5. Interesting idea. Looks like currently the existing functions are hard to find.
Yes. One can say “learn it” - why duplicate what is already in Jmix Studio IntelliJ panel, there are changelogs, Datastore menu, entity designer … a bit dispersed if you ask me. These dispersed menus should stay where they are, but IMO a new management window is needed to aggregate all that, and if it has a tree structure that repeats what is already in Data store and Data model, so what, it’s in one place nicely, and studio code that fetches and displays such data can be reused to extent. It would also be cool if there were a panel in IntelliJ like Gradle, or Database. Can this then be reused for JPA Buddy too? Some sort of collapsible tabular tree view showing a list of entities and their attributes, and then a column with an inline editing combobox showing the current DDL generation status, toggled standard preconditions and such, then another view with changelogs - project changelogs, platform/addon changelogs… panel showing databasechangelock table data …

4. Not sure Studio can distinguish what was manually changed
At first I thought to parse the changelog text and find content that was not autogenerated (I know), but then I thought more about IntelliJ IDE framework, it must have some events that can be handled:

  1. user opens the changelog, it displays it in its editor tab
  2. user closes the tab without changes, or doesn’t save changes - nothing
  3. user edits the changelog, writes something, saves - is it different than original - save, add some icon to changelog entry in the tree or something, higlight …

2. Do you mean when editing the generated changelog? Just a quick way of adding these preconditions instead of writing them manually?
Yes - “toggle if not exists” quick - these are “usual” preconditions, easily generated :slight_smile: and it builds on that thread functionality, later on it can be expanded to some sort of code snippets for preconditions if needed, but just “if not exist” button to add-remove this on table, and column would be fantastic

1. The “Generate Liquibase Changelog” action in the data store context menu does exactly what you
Yes. This is probably misleading me, I want to “view” not generate :slight_smile:

Kind regards,
Mladen

Hello

In my case there is the following scenario.

There is software growing over time. Some clients have the version 1.0 installed, others the Version 1.2 and some 2.0.

Now each of these clients might now install version 3.0 of the software.

And liquibase should check if a certain table ( which was introduced in Version 1.1 ) is already existing. If not, this table should be created. If there is already existing, no error message, move on.

If there is a field missing in a table or the type of the field is wrong; change it, otherwise move on.

Don’t delete any tables. This is really a manual task. Perhaps it is a backup …

I think this is a standard procedure which is really not complicated. The approach as it is actually with checksums which we have to change in case we change any liquibase generated xml is not practicable.

The programmer need full control over the generating and needs his manual changes for each version untouched.

Best regards

Felix

BTW

As MS-SQL does not support the schema per session, the generator which is creating the xml-files should include the defaut schema in the files.

<changeSet id="1" author="author" schemaName="<your_schema_name>">
1 Like