This is the next post about creating my own appview. You can see the first post here: https://willdot.leaflet.pub/3mud5fa247s2e
That post went over how I had implementing indexing the data from the firehose or Jetstream in my case. It wasn't the final shape of the data I needed because I wasn't sure exactly what data I needed to return for a timeline. I soon found out though and I wasn't far off to be honest.
But before I got to the timeline I attempted to try a smaller endpoint because I knew that jumping into the deep end with the timeline, which will contain posts and a lot of data to manage. All that data would then need to fit into a certain shape for the timeline response and I knew it would be tricky. Well here is where I fucked up. Well not really because it meant I just jumped into the deep end anyway and to be honest, I've always thrived in that kinda scenario.
I attempted the app.bsky.actor.getProfile endpoint which I guessed would be a case of returning handles, DIDs and what ever a use has in their profile. Turns out it was all of those things but a lot more!
Here is the documentation for this endpoint and as you can see it's quite a lot of info. However I soon realised that I didn't actually need all of that info and only the Handle and DID were required. Everything else is optional. However things like description and display name I knew were accessible from a users PDS under the app.bsky.actor.profile lexicon record using the self rkey. To do that I would need to go to the users PDS to get that record.
This is where it got tricky and put everything I had learnt in Rust to the test. The request would pass an actor query param which would either be a users handle or their DID. It would have been amazing if it were both because I could have then just returned the response using those values, but unfortunately it wasn't like that.
I knew that I would have to resolve that actor param to get the actors DID because that's what I would need to do any lookups of the user should I need to go to their PDS to get more profile information (spoiler...I was right).
Luckily I found some code that handled exactly what I needed in using the Jacquard library. If input was a DID, return that, otherwise use a resolver to resolve a Handle to a DID. The library handles all of that complex lookup logic for me which was a relief because I've done that in Go before and while It's not hard, I think it would be scary in a Language I'm not overly familiar with.
Once I had the DID, I knew that I now needed the users PDS endpoint which had me stumped. I've done this before but forgot where it came from. I looked back through some of my previous projects and was reminded that a users DID doc contained that information, which seems so obvious now.
Luckily, once again the Jacquard library had code to handle that for me, so it wasn't actually too bad to get that info. Once I had the DID doc, I had the PDS endpoint and with that I could build up an XRPC request to fetch the app.bsky.actor.profile record for that user.
Again the Jacquard library had code for handling this, but I found I was battling Rusts type system quite badly here and a couple times I almost waved the white flag. But a lot of spaghetti code, printing log lines I managed to get back the profile record and get hold of all the information that I needed.
Then all that was left was to build up the app.bsky.actor.getProfile response and I was pretty damn proud of myself.
Then it occurred to me. Other than making a manual HTTP request using my API client of choice, Yaak, I wasn't sure how to test this for real. I knew that there were alternate apps such as Witchsky which allowed you to set a custom Appview, but for some reason it wouldn't accept my Appview, stating an XRPC endpoint was not implemented but I had no idea which one. Then I thought I would check out how Why tested Konbini and there in the Readme was quite clear instructions on how to run the Bluesky client locally and point at my Appview. I did just that, logged in and went to my profile and it worked! It was slow, but it worked!
Why was it slow? Well each time I'm looking at a users profile, it was making an HTTP call to a PDS to get a profile record, which isn't very efficient. But profiles don't tend to change that often, so for my fun little project I can cache that in my Sqlite database. I created a profiles table that would contain the users DID for lookup and then the raw blob response that their PDS returned to me. That way, when viewing a users profile,I can get it from the cache if I have it and decode it and if I don't have it, fetch from their PDS and then cache it after.
Turns out, this came in very useful for when I eventually implemented the timeline endpoint, because each post record contains an author field which uses some of the information from that profile. So if I'm loading 30 timeline posts at once, and worst case each post has a different author, that was 30 profiles being looked up!
Spoiler for the timeline blog which will follow shortly, that scenario was still not efficient enough. First time loading a timeline I would have to fetch everyone's profile. The next time, I would have to fetch more. Eventually it got quicker but only after I had a load of cached records.
I then decided to do it as part of the indexing. When I index a post, I would also grab the users profile of the posts author and store that too. Then I was guaranteed that when loading the timeline, I would have the users profile.
I was happy with my progress. I had implemented my first XRPC endpoint and returned the basic information that I needed. Sure that are other fields that I should return such as follower count (which I will do eventually via Constellation backlinks) and post count which I haven't decided on yet. But for now I was happy and was impressed at how much I had actually learnt about how to use Rust. Sure the type system kicked my ass quite a lot, but I got there eventually.
Next up, was the app.bsky.feed.getTimeline endpoint which as I previously mentioned, was sure would be a lot more complex. Look out for that blog post and once again, check out the code for this project here