Feature Request: make result row limit configurable (was: Feature Request: Display results in pages)

[csubhodeep]

Hello DuckDB UI Team,

For some of the queries that can potentially return a large number of records. It would be nice to show the results in pages where the maximum number of results displayed per page should be something controllable (with a suitable default, let's say 50) set globally for the session from the UI (or from the CLI using one of the SET statements).

Currently not having this significantly slows down the cell execution - probably not because of duckdb backend query execution but because the results need to be transported to the frontend.

Conceptual solution - You could internally use the rowid of the final result, assign a "page number" to each row id and then based on the page of the result that the user wants to view - filter for that page. If the whole result set is created as a temporary view, then this basically would mean that a new SELECT statement is executed with a WHERE to calculate the next set of records on the fly in chunks each time the user clicks next page.

[jraymakers]

The UI limits the number of results for any user-run query to 50,000 rows. This reduces the amount of transport and processing to a manageable amount. Furthermore, the UI virtualizes the display of results, in both the rows and columns, so a much smaller amount of data has to actually be rendered.

We're always working on improving how queries are run, and may make the above limit more flexible in the future. Of course, you can always use LIMIT and OFFSET, possibly combined with using CREATE TABLE to save the results to a table, to control the amount of data processed yourself.