Paginate superuser users/projects lists with server-side search#4474
Paginate superuser users/projects lists with server-side search#4474sakibsadmanshajib wants to merge 9 commits intoOpenFn:mainfrom
Conversation
sakibsadmanshajib
force-pushed
the
paginate-superuser-list
branch
from
a7f7859 to
fcf4b25
Compare
There was a problem hiding this comment.
Pull request overview
This PR improves superuser admin list performance by moving /settings/users and /settings/projects filtering/sorting/pagination from in-memory operations to database-backed queries with URL-param-driven state, and adds trigram indexes to support efficient search.
Changes:
- Added
Accounts.list_users_for_admin/1andProjects.list_projects_for_admin/1usingRepo.paginate/2with normalized params for safe filtering/sorting. - Updated Users/Projects LiveViews to treat URL params (
page,filter,sort,dir,page_size) as the source of truth and to use Scrivener pages in table rendering. - Added
pg_trgm+ GIN trigram indexes for key search columns, plus tests covering pagination and param safety.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/lightning_web/live/user_live_test.exs | Adds LiveView pagination navigation coverage for users index. |
| test/lightning_web/live/project_live_test.exs | Adds LiveView pagination navigation coverage for projects index. |
| test/lightning/projects_test.exs | Adds coverage for admin projects search/sort and invalid param fallback. |
| test/lightning/accounts_test.exs | Adds coverage for admin users search/sort and invalid param fallback. |
| priv/repo/migrations/20260226162000_add_admin_search_trgm_indexes.exs | Adds pg_trgm extension and trigram indexes for admin search paths. |
| lib/lightning_web/live/user_live/table_component.ex | Switches users table component to Scrivener-page-driven rendering + URL patching. |
| lib/lightning_web/live/user_live/index.html.heex | Passes normalized table_params down to the table component. |
| lib/lightning_web/live/user_live/index.ex | Normalizes/validates table URL params (sort/dir/page/page_size/filter). |
| lib/lightning_web/live/user_live/components.ex | Updates users table component API to accept page + pagination_path. |
| lib/lightning_web/live/project_live/index.html.heex | Switches projects table to use Scrivener page entries + pagination URL. |
| lib/lightning_web/live/project_live/index.ex | Adds param normalization and uses Projects.list_projects_for_admin/1 for server-side table data. |
| lib/lightning/projects.ex | Implements paginated/sortable/searchable admin projects query. |
| lib/lightning/accounts.ex | Implements paginated/sortable/searchable admin users query. |
| docs/plans/2026-02-26-superuser-users-projects-pagination-search-design.md | Documents design/approach and rollout considerations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
midigofrank
left a comment
midigofrank
left a comment
There was a problem hiding this comment.
Hey @sakibsadmanshajib , great job here. The superuser interface needs some love, thank you for picking this up.
I have left some comments:
- I don't see a need for indexes on those tables. They're usually tiny
- Consider using using an embedded schema to have a single place for validations, like: https://github.com/OpenFn/lightning/blob/main/lib/lightning/workorders/search_params.ex
- With the embedded schema in place, you can try to refactor the list_users function, like: https://github.com/OpenFn/lightning/blob/main/lib/lightning/invocation.ex#L482
There was a problem hiding this comment.
Let's leave the indexes out for now, we can always add them later when needed.
There was a problem hiding this comment.
You could leave the parsing and default values to an ecto schema. Have a look at: https://github.com/OpenFn/lightning/blob/main/lib/lightning/workorders/search_params.ex and see how the runs index page utilizes ecto changesets to handle the filtering
| server-side filtering and sorting. | ||
| """ | ||
| @spec list_users_for_admin(map()) :: Scrivener.Page.t() | ||
| def list_users_for_admin(params \\ %{}) do |
There was a problem hiding this comment.
You could have the normalization logic by having an embedded schema and doing validations using ecto changesets. Have a look at: https://github.com/OpenFn/lightning/blob/main/lib/lightning/invocation.ex#L482 for inspiration
Description
This PR adds server-side pagination, sorting, and search to the superuser users/projects settings list views, and moves those list operations out of in-memory filtering into SQL-backed queries.
Closes #2913
Validation steps
mix formatmix test test/lightning/accounts_test.exs test/lightning/projects_test.exs test/lightning_web/live/user_live_test.exs test/lightning_web/live/project_live_test.exs --seed 0mix test test/lightning_web/live/audit_live_test.exs --seed 0Additional notes for the reviewer
20260226162000_add_admin_search_trgm_indexes.exs.page,filter,sort,dir) for both users and projects.AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
Pre-submission checklist
/reviewwith Claude Code):owner,:admin,:editor,:viewer)