Change in cache doesn't propagate

[LiLatee]

Hey! 👋

I have such a case. I have a request let's say GpostsReq. I use that request with CacheAndNetwork FetchPolicy. But for pull to refresh action I put into Client.requestController.add(modifiedRequest) modified request which has set NetworkOnly FetchPolicy, so I display results only from network.
And let's check the following steps:

1. Like a post using another mutation. This mutation modifies cache and that change propagates for GpostsReq so I can see that my post has been liked.
2. Then make a pull to refresh with modified request which has set NetworkOnly FetchPolicy.
3. Like another post. Now post doesn't update. It looks like now my query doesn't involve changes from cache. But that wasn't an intention.

The same story with fetchMore action for which I have set NoCache FetchPolicy.

Am I missing something or that's expected behaviour? 😞

[knaeckeKami]

Hm, it could be that the refetch indeed causes the stream to stop watching the cache, since it has a no cache fetch policy.

But if you watch the cache anyway, you don't need to use the request controller anyway - just execute (client.request()) the pull to refresh request with .NetworkOnly, this will update the cache, and the cache will refresh the original request that uses .CacheAndNetwork.

[knaeckeKami]

But no, this is not expected behaviour, and this test: https://github.com/gql-dart/ferry/blob/master/packages/ferry/test/typed_links/request_controller_typed_link_test.dart#L251 should ensure it does not happen.

[LiLatee]

But if you watch the cache anyway, you don't need to use the request controller anyway - just execute (client.request()) the pull to refresh request with .NetworkOnly, this will update the cache, and the cache will refresh the original request that uses .CacheAndNetwork.

It also doesn't work. I had to change requestId to another one and then it worked (either with with use ofrequestController or request) 🤔 So it looks like the problem is that both request and modifiedRequest have the same requestId

[LiLatee]

But no, this is not expected behaviour, and this test: master/packages/ferry/test/typed_links/request_controller_typed_link_test.dart#L251 should ensure it does not happen.

But that test doesn't use no cache fetchPolicy 🤔

[LiLatee]

So it looks like the problem is that both request and modifiedRequest have the same requestId

But unfortunately it won't solve the issue with fetchMore because it has to have the same requestId in order to merge results into a single list if I am not wrong.

[knaeckeKami]

Yes, but is also does not even use a cache, it just tests the request controller typed link. It's just an assertion that the previous stream is continued to be listened.

similar tests with a full client are below (e.g. https://github.com/gql-dart/ferry/blob/master/packages/ferry/test/typed_links/request_controller_typed_link_test.dart?rgh-link-date=2024-01-30T16%3A51%3A24Z#L408 ), maybe you can craft a reproducible example out of that?

[knaeckeKami]

It also doesn't work. I had to change requestId to another one and then it worked (either with with use ofrequestController or request) 🤔 So it looks like the problem is that both request and modifiedRequest have the same requested

Yes, in that case you can't use the same requestId.

[knaeckeKami]

But unfortunately it won't solve the issue with fetchMore because it has to have the same requestId in order to merge results into a single list if I am not wrong.

If you want caching + pagination to work together, I would recommend just merging all the pages in the cache, and removing the arguments for paging from the cache.

like here: https://ferrygraphql.com/docs/cache-configuration#the-fields-property

alternatively,/additionally you can also add the read function for the type policy and read all the pages there (or just the ones that you want)

[LiLatee]

If you want caching + pagination to work together, I would recommend just merging all the pages in the cache, and removing the arguments for paging from the cache.
like here: ferrygraphql.com/docs/cache-configuration#the-fields-property

I've already tried that some time ago, but I had some problems. I also use updateResult field of OperationRequest and merge results there and because of that, I couldn't find a way to properly merge results. But I will try it tomorrow once again.

maybe you can craft a reproducible example out of that?

I have a list of things that I would try to do regarding ferry, but I need to finish migrating to ferry asap and I need to find some free time for improvements/fixes for ferry in the future 🙏

[LiLatee]

alternatively,/additionally you can also add the read function for the type policy and read all the pages there (or just the ones that you want)

To be honest I am a little bit confused and don't remember right now when to use read, merge and updateResult, but I will take a look 🤞

[knaeckeKami]

I think, what you want is :

To set keyArgs for the type policy to an empty array. This means, the Posts request will be cached like this:

{
"Query": 
   "posts": [{...}, {...}] // just a single list posts
}

instead of

{
"Query":
   "posts(offset: 0, limit: 3)" : [{...}, {...}],
   "posts(offset: 3, limit: 3)" : [{...},  {...}],
}

So that there is just a single entry for the query.

The default behaviour of the cache is, that when A new result for the same query comes in,
that the old items are replaced.

We now want to change that, so that the previous and next result are mered.
You can achieve that by using the merge function as in the docs.

Now, if you watch the cache for the posts query, it will always return all cached results.

You can fetch more or refetch by just executing more requests, no need for any requestId or fetchMore params, or .requestController.add().

And mutations that update entries (like your "like" example would continue to just work.

You probably don't need to implement the read function in the type policy.

[LiLatee]

Hmm, right now I have

{
      'posts': FieldPolicy(
        keyArgs: ['userId'],
      ),
}

I will try your recommendation tomorrow, but as I understand I have to modify the merge function from docs because it assumes that existing and incoming are lists, but in my case they are maps and inside these maps I have a list with posts. I will let you tomorrow.

And btw many thanks for your help 🙏

[knaeckeKami]

Yes, if they are maps, you should use a LinkedHashMap instead of the LinkedHashSet.

btw, here are tests that are close to your scenario:

ferry/packages/ferry/test/field_policy/field_policy_test.dart

Line 11 in 7f6a63c

test('can merge pages in cache using custom field policy', () async {

[LiLatee]

To set keyArgs for the type policy to an empty array. This means, the Posts request will be cached like this:
...
So that there is just a single entry for the query.

Yes that is basically something I want to achieve and I already had that with a small difference that in keyArgs I have defined userId param because I want to differentiate caches for different users' timelines.

And these typePolicies should be put inside Cache class? I see Client class has also typePolicies field

But I don't get the following.

Now, if you watch the cache for the posts query, it will always return all cached results.

Do I have to do something to watch the cache?

You can fetch more or refetch by just executing more requests, no need for any requestId or fetchMore params, or .requestController.add().

So the first call of my request works fine, it executes the merge function. Unfortunately, after calling once again the same query (in order to make pull to refresh) it doesn't make any request (I cannot spot it in devtools).
Ok... And now I noticed that difference between original request and pull to refresh query is that pull to refresh one has NetworkOnly and original one has CacheAndNetwork. After setting both to CacheAndNetwork pull to refresh worked fine. But I would prefer to have NetworkOnly in the pull to refresh 🤔

Similar story with fetchMore, but in that case I have to set after parameter (indicating the page to fetch) and because of that difference with original request that request is not called. In your test you modifies the offset var and everything seems to work fine 🤔

And what about updateResult field of a request? Do I need it? 🤔

[knaeckeKami]

Do I have to do something to watch the cache?

watching a stream with .CacheFirst or .CacheAndNetwork is enough.

you can also watch the cache directly using client.cache.watchQuery();

Do I have to do something to watch the cache?

yes, you would add the typepolicies map in the constructor of the cache.

Ok... And now I noticed that difference between original request and pull to refresh query is that pull to refresh one has NetworkOnly and original one has CacheAndNetwork. After setting both to CacheAndNetwork pull to refresh worked fine. But I would prefer to have NetworkOnly in the pull to refresh 🤔

how do you execute the pull to refresh and the fetchmore request?

And what about updateResult field of a request? Do I need it? 🤔

no, if you do do merging in the cache and watch the cache, you don't need requestId and updateResult.