Export all rows pagination issue

Hi,

I had Export button on workbench and when i click on export button and select all rows it exports all data from first page to last page.
But if i am on last page on in middle page and click on export it exports from last page if i am on last page or from middle page from where i am clicking on export all rows.
Is it possible not matter from which page user is trying to export data records will be generated from first page to last page.

Any solution for this ?

Greetings,

Can you provider more details what do you use to export (Grid Export actiona addon?) and little bit more details how to reproduce this behaviour?

Best regards,
Dmitry

When a user navigates to the screen .The “Export” functionality, when used with the “All rows” option, is behaving inconsistently depending on the user’s current page of records.

When the user navigates to the last page of records on the screen and selects the “Export” option with the “All rows” setting, the downloaded file incorrectly contains only the records visible on that last screen.

Conversely, when the same “Export” with “All rows” action is performed while the user is on the first page of the report records, the system correctly exports all records as intended.

Below code is used

<button id="exportTable" caption="Export Report">
which extends ExportAction

image

Hello again,

I tested this on Jmx 1.5.5 and there is bug reproduced. What is your version of Jmix?
There was several bugs related with Jmix 1.5+ and 2.3+, e.g.
export issues (many)

Try use Jmix 1.6.2 instead, where this issues fixed (i tested it and no errors in Jmix 1.6.2)

Otherwise, as work-around, you can override Export action and use on your own, but simplest solution is to migrate to latest jmix 1.x version with bugfixes

P.S. If you would still have an errors that still reproduce, please share demo project and steps to reporduce and we will fix it

Best regards,
Dmitry

1 Like

Currently i am using
version '1.5.0'

I will fix this with migration also can you help me with the steps to override Export action and use on our own logic.

Hi @d.cherkasov
Any Help on Custom override Export action steps

Hi,

Yes, easiest way to extend existing one and override own logics:

Action:

@StudioAction(target = "io.jmix.ui.component.ListComponent", description = "Exports selected entities to JSON with custom loading logic")
@ActionType(CustomJsonExportAction.ID)
public class CustomJsonExportAction extends JsonExportAction {

    public static final String ID = "customJsonExport";
    private ApplicationContext applicationContext1;

    public CustomJsonExportAction(String id) {
        this(id, null);
    }

    public CustomJsonExportAction() {
        this(ID);
    }
    public CustomJsonExportAction(String id, String shortcut) {
        super(id, shortcut);
    }

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) {
        applicationContext1 = applicationContext;
        super.setApplicationContext(applicationContext);
        withExporter(CustomJsonExporter.class);
    }

    @Override
    public <T> T withExporter(Class<T> exporterClass) {
        var exporter = (TableExporter) applicationContext1.getBean("custom_JsonExporter");
        setTableExporter(exporter);
        return (T) exporter;
    }
}

Exporter:

@Component("custom_JsonExporter")
@Scope(BeanDefinition.SCOPE_PROTOTYPE)
public class CustomJsonExporter extends JsonExporter {

    @Qualifier("customExecutor")
    @Autowired
    private ExecutorService executorService;

    private final AllEntitiesLoaderFactory allEntitiesLoaderFactory;

    public CustomJsonExporter(Metadata metadata, AllEntitiesLoaderFactory allEntitiesLoaderFactory) {
        super(metadata, allEntitiesLoaderFactory);
        this.allEntitiesLoaderFactory = allEntitiesLoaderFactory;
    }

    @Override
    protected Collection<Object> getItems(DataGrid<Object> dataGrid, ExportMode exportMode) {
        // OR your custom logics to load entities from grid

        return ExportMode.CURRENT_PAGE == exportMode
                ? dataGrid.getItems().getItems().collect(Collectors.toList())
                : dataGrid.getSelected();
    }

    @Override
    protected Collection<Object> getItems(Table<Object> table, ExportMode exportMode) {
        // OR your custom logics to load entities from table

        return ExportMode.CURRENT_PAGE == exportMode
                ? table.getItems().getItems()
                : table.getSelected();
    }

    @Override
    public void exportDataGrid(Downloader downloader, DataGrid<Object> dataGrid, ExportMode exportMode) {
        if (exportMode == ExportMode.ALL_ROWS) {
            exportDataGridAll(downloader, dataGrid, exportMode);
        } else {
            exportDataGridSelected(downloader, dataGrid, exportMode);
        }
    }

    private void exportDataGridAll(Downloader downloader, DataGrid<Object> dataGrid, ExportMode exportMode) {
        Gson gson = createGsonForSerialization();
        JsonArray jsonElements = new JsonArray();

        CompletableFuture.runAsync(() -> {
                    AllEntitiesLoader entitiesLoader = allEntitiesLoaderFactory.getEntitiesLoader();
                    entitiesLoader.loadAll(dataGrid.getItems(),
                            context -> {
                                JsonObject jsonObject = createJsonObjectFromEntity(dataGrid, context.getEntity());
                                jsonElements.add(jsonObject);
                                return true;
                            },
                            ExporterSortHelper.getSortOrder(dataGrid.getSortOrder()));
                }, executorService
        ).thenAccept(e -> {
            downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                            uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                    getFileName(dataGrid) + ".json", DownloadFormat.JSON);

            super.exportDataGrid(downloader, dataGrid, exportMode);
        });
    }

    private void exportDataGridSelected(Downloader downloader, DataGrid<Object> dataGrid, ExportMode exportMode) {
        Gson gson = createGsonForSerialization();
        JsonArray jsonElements = new JsonArray();

        Collection<Object> items = getItems(dataGrid, exportMode);

        for (Object entity : items) {
            JsonObject jsonObject = createJsonObjectFromEntity(dataGrid, entity);
            jsonElements.add(jsonObject);
        }

        downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                        uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                getFileName(dataGrid) + ".json", DownloadFormat.JSON);

        super.exportDataGrid(downloader, dataGrid, exportMode);
    }

    @Override
    public void exportTable(Downloader downloader, Table<Object> table, ExportMode exportMode) {
        // same as grid

        super.exportTable(downloader, table, exportMode);
    }

    @Override
    protected JsonObject createJsonObjectFromEntity(DataGrid<Object> dataGrid, Object entity) {
        // override if you want custom json creation for dataGrid
        return super.createJsonObjectFromEntity(dataGrid, entity);
    }

    @Override
    protected JsonObject createJsonObjectFromEntity(Table<Object> table, Object entity) {
        // override if you want custom json creation for Table
        return super.createJsonObjectFromEntity(table, entity);
    }
}

Usage (same as existing actions):

<action id="customJsonExport" 
                        type="customJsonExport"
                        caption="Export to JSON"
                        icon="font-icon:FILE_CODE_O"/>

<button id="customJsonExportBtn" action="usersTable.customJsonExport"/>

Best regards,
Dmitry

Hi,

Cannot resolve symbol ‘AllEntitiesLoaderFactory’ unable to find import.

Hi,

I had override the TableExporter can you help on what need to be change in logic to execute export from first page result instead of exporting from the selected page.


@Component("customReportExpoter")
@Scope(BeanDefinition.SCOPE_PROTOTYPE)
public class CustomReportExporter extends AbstractTableExporter<ExcelExporter> implements TableExporter {
}

Hi

Better to not override logics by following reasons:

  1. Table have own sort that not comparing to SQL-like sort (its likely random 50 rows per page, each sorted in memory, but no guarantee that jpql loaded that data struct strictly depending on sql logics)
  2. There is hard to maintain or own logics of selection that “first” page. E.g. First page sorted by name in table != First page sorted by name in SQL.
  3. We cant load first page of table without moving client to the first page of table.

But, if you still want to load only “first page” instead of current, there is most likely logics for Exporter:

package com.company.jmixcustomexport.export;

import com.company.jmixcustomexport.entity.User;
import com.company.jmixcustomexport.screen.user.UserBrowse;
import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import io.jmix.core.DataManager;
import io.jmix.core.Metadata;
import io.jmix.core.querycondition.Condition;
import io.jmix.gridexportui.exporter.ExportMode;
import io.jmix.gridexportui.exporter.ExporterSortHelper;
import io.jmix.gridexportui.exporter.entitiesloader.AllEntitiesLoader;
import io.jmix.gridexportui.exporter.entitiesloader.AllEntitiesLoaderFactory;
import io.jmix.gridexportui.exporter.json.JsonExporter;
import io.jmix.ui.component.DataGrid;
import io.jmix.ui.component.Table;
import io.jmix.ui.component.impl.DataGridSettingsUtils;
import io.jmix.ui.download.ByteArrayDataProvider;
import io.jmix.ui.download.DownloadFormat;
import io.jmix.ui.download.Downloader;
import io.jmix.ui.model.DataComponents;
import io.jmix.ui.model.DataLoader;
import io.jmix.ui.model.ScreenData;
import io.jmix.ui.screen.StandardLookup;
import io.jmix.ui.screen.UiControllerUtils;
import io.jmix.ui.sys.UiControllerMeta;
import io.jmix.ui.sys.UiControllerReflectionInspector;
import io.jmix.ui.sys.UiDescriptorUtils;
import io.jmix.ui.widget.AppUIUtils;
import io.jmix.uidata.filter.UiDataFilterMetadataTools;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.context.annotation.Scope;
import org.springframework.stereotype.Component;

import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.stream.Collectors;

import static liquibase.repackaged.net.sf.jsqlparser.util.validation.metadata.NamedObject.table;

/**
 * Custom JSON exporter implemented as a Spring Bean component.
 * This class handles the export of entities to JSON format with custom entity loading logic.
 */


@Component("custom_JsonExporter")
@Scope(BeanDefinition.SCOPE_PROTOTYPE)
public class CustomJsonExporter extends JsonExporter {

    private final DataManager dataManager;
    @Qualifier("customExecutor")
    @Autowired
    private ExecutorService executorService;

    private final AllEntitiesLoaderFactory allEntitiesLoaderFactory;

    public CustomJsonExporter(Metadata metadata, AllEntitiesLoaderFactory allEntitiesLoaderFactory, DataManager dataManager) {
        super(metadata, allEntitiesLoaderFactory);
        this.allEntitiesLoaderFactory = allEntitiesLoaderFactory;
        this.dataManager = dataManager;
    }

    @Override
    protected Collection<Object> getItems(DataGrid<Object> dataGrid, ExportMode exportMode) {
        // OR your custom logics to load entities from grid
        return ExportMode.CURRENT_PAGE == exportMode
                ? dataGrid.getItems().getItems().collect(Collectors.toList())
                : dataGrid.getSelected();
    }

    @Override
    protected Collection<Object> getItems(Table<Object> table, ExportMode exportMode) {
        // OR your custom logics to load entities from table
        return ExportMode.CURRENT_PAGE == exportMode
                ? table.getItems().getItems()
                : table.getSelected();
    }

    @Override
    public void exportDataGrid(Downloader downloader, DataGrid<Object> dataGrid, ExportMode exportMode) {
        if (exportMode == ExportMode.ALL_ROWS) {
            exportDataGridAll(downloader, dataGrid, exportMode);
        } else {
            exportDataGridSelected(downloader, dataGrid, exportMode);
        }
    }

    private void exportDataGridAll(Downloader downloader, DataGrid<Object> dataGrid, ExportMode exportMode) {
        Gson gson = createGsonForSerialization();
        JsonArray jsonElements = new JsonArray();

        CompletableFuture.runAsync(() -> {
                    AllEntitiesLoader entitiesLoader = allEntitiesLoaderFactory.getEntitiesLoader();
                    entitiesLoader.loadAll(dataGrid.getItems(),
                            context -> {
                                JsonObject jsonObject = createJsonObjectFromEntity(dataGrid, context.getEntity());
                                jsonElements.add(jsonObject);
                                return true;
                            },
                            ExporterSortHelper.getSortOrder(dataGrid.getSortOrder()));
                }, executorService
        ).thenAccept(e -> {
            downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                            uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                    getFileName(dataGrid) + ".json", DownloadFormat.JSON);

            super.exportDataGrid(downloader, dataGrid, exportMode);
        });
    }

    private void exportDataGridSelected(Downloader downloader, DataGrid<Object> dataGrid, ExportMode exportMode) {
        Gson gson = createGsonForSerialization();
        JsonArray jsonElements = new JsonArray();

        Collection<Object> items = getItems(dataGrid, exportMode);

        for (Object entity : items) {
            JsonObject jsonObject = createJsonObjectFromEntity(dataGrid, entity);
            jsonElements.add(jsonObject);
        }

        downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                        uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                getFileName(dataGrid) + ".json", DownloadFormat.JSON);

        super.exportDataGrid(downloader, dataGrid, exportMode);
    }

    @Override
    public void exportTable(Downloader downloader, Table<Object> table, ExportMode exportMode) {

        var res2 = UiControllerUtils.getScreen(table.getFrame().getFrameOwner());


        if(res2 instanceof UserBrowse) {

            ScreenData screenData = UiControllerUtils.getScreenData(table.getFrame().getFrameOwner());

            DataLoader usersDl = screenData.getLoader("usersDl");


            List<User> userList = dataManager.load(User.class)
                    .condition(Objects.requireNonNull(usersDl.getCondition())) // same condition as in the screen
                    .sort(ExporterSortHelper.getSortOrder(table.getSortInfo()))
                    .maxResults(50) // always one page
                    .firstResult(0) // always first page
                    .list();

            Gson gson = createGsonForSerialization();
            JsonArray jsonElements = new JsonArray();
            for (Object entity : userList) {
                JsonObject jsonObject = createJsonObjectFromEntity(table, entity);
                jsonElements.add(jsonObject);
            }

            downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                            uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                    getFileName(table) + ".json", DownloadFormat.JSON);
            return;
        }


        if (exportMode == ExportMode.ALL_ROWS) {
            exportTableAll(downloader, table, exportMode);
        } else {
            exportTableSelected(downloader, table, exportMode);
        }
    }

    private void exportTableAll(Downloader downloader, Table<Object> table, ExportMode exportMode) {
        Gson gson = createGsonForSerialization();
        JsonArray jsonElements = new JsonArray();

        CompletableFuture.runAsync(() -> {
                    AllEntitiesLoader entitiesLoader = allEntitiesLoaderFactory.getEntitiesLoader();
                    entitiesLoader.loadAll(table.getItems(),
                            context -> {
                                JsonObject jsonObject = createJsonObjectFromEntity(table, context.getEntity());
                                jsonElements.add(jsonObject);
                                return true;
                            },
                            ExporterSortHelper.getSortOrder(table.getSortInfo()));
                }, executorService
        ).thenAccept(e -> {
            downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                            uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                    getFileName(table) + ".json", DownloadFormat.JSON);
        });
    }

    private void exportTableSelected(Downloader downloader, Table<Object> table, ExportMode exportMode) {
        Gson gson = createGsonForSerialization();
        JsonArray jsonElements = new JsonArray();

        Collection<Object> items = getItems(table, exportMode);

        for (Object entity : items) {
            JsonObject jsonObject = createJsonObjectFromEntity(table, entity);
            jsonElements.add(jsonObject);
        }

        downloader.download(new ByteArrayDataProvider(gson.toJson(jsonElements).getBytes(StandardCharsets.UTF_8),
                        uiProperties.getSaveExportedByteArrayDataThresholdBytes(), coreProperties.getTempDir()),
                getFileName(table) + ".json", DownloadFormat.JSON);
    }

    @Override
    protected JsonObject createJsonObjectFromEntity(DataGrid<Object> dataGrid, Object entity) {
        // override if you want custom json creation for dataGrid
        return super.createJsonObjectFromEntity(dataGrid, entity);
    }

    @Override
    protected JsonObject createJsonObjectFromEntity(Table<Object> table, Object entity) {
        // override if you want custom json creation for Table
        return super.createJsonObjectFromEntity(table, entity);
    }
}

Test project (jmix 1.6):

jmixcustomexport.zip (285.1 KB)

This is example, how you can “imitate” first page load logics. Its ok for filter, but again sort - is problem. Or, otherwise, instead of in-memory logics you can duplicates sort by in JPQL or loader in screen and then pages would be same. (force avoid jmix table in-memory sorter)

By the way, i won’t recommend maintain own logics and “generic” solution by you own. If you still want this solution, check our Jmix Assistant, probably it would helps you faster.

Best regards,
Dmitry