Paging Library

Contents of the paging library
Introduction
Recyclerview Introduction
Issue with a large number of data sets in recycler view
How paging helps in this
Paging elements
Data source types
Issues associated with earlier way of endless loading
Using paging instead of endless list functionality
Using Paging with -> Local + Remote
Blocks of paging component

---

Blocks of paging component
Data Source
Using the paging source
Challenges faced without a paging library
Paging Data
Config involved in paging data
How the data stream from pager is rendered in UI(RecyclerView)
How to display header and footer to for the recycler-view
Transforming the data

Introduction

• Our world has become overloaded with data.
• data include things like news feed, video, music and more...
• When things keep increasing we need to keep in mind not to overload the device itself because the device has limited number of resources.
• Phone has a very limited amount of real estate available to show the data that is available.

---

Recyclerview Introduction

• On this limited amount of real estate, displaying the data in an efficient manner was solved by the usage of the view holder pattern in android.
• Instead of created n-views for n-number of data, using the recycler view we could create few views for n-number of data and recycle the views as we scroll along.
• We can see how this is done in the diagram above.

---

Issue with a large number of data sets in recycler view

• The view sure is recycled and helps in smooth scrolling but the data set is still large this can be handled efficiently.
• But still the data is in memory say a list. If the data set increases to tens and thousands of items, we may run into performance issues.
• This performance issue is for sure in older android devices.

---

How paging helps in this

• Paging library helps in fixing this issue in the data side by displaying only the data needed currently on the screen display.
• What view holder does on the view side, the paging does on the data side
• Paging library doesn't keep all the data in the memory at once - > Paging library pulls the data from the data source on an as-needed basis.
• Paging library is designed to particularly operate well on live data and room database

---

Paging elements

Paging element	Description
Data Source	Determines where data comes from, like if it is room database or a network
PageList	It is like a window to the list of items
PageList Adapter	It is a subclass of recycler view, that is used to show paged list in recycler views

---

Data source types

Data source type	Description
Page Keyed Data Source	It gives paged items at a time
Item Keyed Data Source	It gives items one by one
Positional Data Source	Any number of items from any position

---

Issues associated with earlier way of endless loading

1. We need to keep track of keys so that we can retrieve the next and previous pages.
2. We need to request the correct next page when the user scrolls at the end of the recycler view.
3. We need to prevent duplicate requests.

---

Using paging instead of endless list functionality

• Paging library takes care of all the three points in the issues mentioned.
• Paging keeps track of the loading state, and if there is an error in loading, we can show a view with retry view.
• Refreshing of the list is also just a method call involved.
• We can transform the data using the list transformations using operators like filter, map etc which can be flow or rx-java.
• Adding the list separators is also very much easier.

---

Using Paging with -> Local + Remote

---

Blocks of paging component

Data Source

• Data source is a single source of data - it's the single source of truth.
• If you are loading the data from a single source, Be it from file-system, remote-source, local-database, we implement the paging source
• If we are using the room, it implements the paging source to help you.
• If you are fetching the data from the server and storing it locally finally displaying the data to the end-user, We use remote-mediator that handles the data from the server and maintains a cached local state.

Using the paging source

• To use the paging source we need to extend the PagingSource class and override the load method which is a suspend function.
• Since this is a suspend function, we can use other suspend functions inside this like getting data from local database, remote-api, etc.
• Here we can pass the prev-page, next-page keys received from the server.
• If we don't receive the keys and the API is index-based, we just use calculate it here.
• Here we return data along with keys wrapped in load-result, And we return the error if there is an error occurred.
• Because of this when you build a network request, the keys are provided.

Challenges faced without a paging library

• Many times when we need offline support for data once it is retrieved from the server. We call it a layered data source which is a combination of network and local data source.
• Using the paging this works great together, with fresh data always from the server and sync data retrieved from the local database.
• This also offers offline support.
• Without the paging, it's hard to achieve and prone to bugs.

• Challenges faced include things like using the connected state is very hard since individual requests can succeed or fail.
• Also here we always request data from the network even if the data is available in the cache.
• So a better solution is always considering the database as the source of truth and get the data from the database and on first load or when the database is exhausted as represented below.

Using the Remote Mediator

• As mentioned above if we are using just room the paging source is provided by room by out-of-the-box

 @Query("SELECT * FROM Movie ORDER BY ranking")
  fun allMovies(): DataSource.Factory<Int, Movie>

• If we are using room and a network data source we need to use the remote-mediator class
• We need to use a class that uses remote-mediator class
• Once we do that we need to over-ride the load-method.
• This load-method will be triggered when we require new data.
• The load-method is a suspend function. meaning we can call other suspend functions in it.
• Using the service class reference from the constructor, we can store it in the database.
• In the mediator class, we need to tell the paging if there is more data or not in the API.

val isEndOfThePageAchieved = dataFromApi.isEmpty()
return MediatorResult.Success(isEndOfThePageAchieved)

If there is some error or exception, we return the error using

val isEndOfThePageAchieved = dataFromApi.isEmpty()
return MediatorResult.Error(exception)

Paging Data

• Pager -> This is the primary entry point for the pager
• We construct the paging using the configuration and the data source that makes the pager

When the API is the source of truth

Pager(
       PagingConfig(pageSize=30)
     ){
          CustomPagingSource(apiService)
      }

When we are using a database and API, we consider the database as the source of truth

In this case we need to pass a remote mediator also along with the config to the paging constructor

val sourceOfPaging = databaseService.Dao.List
Pager(
       PagingConfig(pageSize=30)
       RemoteMediator(databaseService,remoteService)
     ){
          sourceOfPaging
      }

Difference you can note in both the above types is as explained in earlier concepts mediator manages the part of handling the caching part, once the data is changed in the database the observers in the UI will be notified

Config involved in paging data

PagingConfig(
              pageSize = 30,
              prefetchDistance = 30,
              enablePlaceHolders = true,
              initialLoadSize = 30*3,
              maxSize = PagingConfig.MAX_SIZE_UNBOUNDED
              jumpThreshold = 0
            )

The Pager returns the flow/observable/flowable/livedata depending on whether we are using coroutines or rxjava this can be used by the UI to display the dat to the end user

How the data stream from pager is rendered in UI(RecyclerView)

• For this purpose we implement paging data adapter
• This paging data adapter extends the regular RecyclerView adapter and provides all the functionalities of recycler view and in addition, it provides ways to handle paging data.
• In the constructor of the Paging data adapter we can even pass diffUitl utility for smoothness to handle abruptness found when an item is removed and notifydatasetchange() is applied.
• We have a coroutine flow in the view that collects the data from the view model and submits it to the pager adapter.

How to display header and footer for the recycler-view

• Since having a header and footer is not part of the actual recycler-view list, paging provides a new adapter for this called LoadStateAdapter
• This adapter will be seperate for both header and footer
• Basically if we have header, footer, and list -> We have three adapters for our recycler-view list.
• We set all the adapters like mentioned below.

private fun setupViews() {
        binding.apply {
            // Adapter: List of items
            rvPosts.adapter = adapter
            // Adapters: Header and Footer item
            rvPosts.adapter = adapter.withLoadStateHeaderAndFooter(
                // Header
                header = RemoteApiLoadingAdapter { adapter.retry() },
                // Footer
                footer = RemoteApiLoadingAdapter { adapter.retry() }
            )
     }
}

Transforming the data

• One additional feature useful is the ability to transform the data before it is displayed in the recycler-view.
• Before it's published we can use map, filter to modify and pass the data to the recycler-view.
• If we are handling the configuration changes, the transformations can be expensive so after the transformations are done and just before publishing data we use cachedin so that data can be reused in adapter instead of applying the transformations again.