#devdiary

Some evenings I only have time to answer questions and comment on pull requests.

Which is kind of cool :P

#opensource #socialhome #devdiary

Jason Robinson

Found good use for the new Diaspora URI scheme. For the federation library I need some kind of global identifier when #ActivityPub support is added. A GUID doesn't work since it doesn't define have a location. I was already writing code for <guid>@<domain.tld> format when I saw the Diaspora URI scheme - which fits this usage perfectly πŸ‘ This gives each entity a globally unique URI ID, regardless of protocol.

#devdiary #federation #diaspora

jaywink/federation
Python library for abstracting social federation protocols

Jason Robinson

Making #Socialhome pre-cache streams content id's in #Redis for each user. Going to have super fast stream loading 😍 Basically something like this:

  • On save -> push to a Redis ordered set for each user for each stream the content ID
  • On stream load, pull out X items from the ordered set in reverse order
  • If we have enough items -> fetch objects from DB and render
  • If we need more items, collect enough using normal queryset filters (slow, current method) -> fetch objects from DB and render

I tried using a Redis List at first, but I couldn't figure out how to make "load more" work with that. If the client says "I want all after item X" then I have to pull out X items from the Redis list after that item. But ordered sets are perfect since they have unique members and items can be neatly pulled out in order from either direction.

Actually the Socialhome streams are not even slow at the moment. But doing this pre-caching allows for any kind of complex calculations to be made for custom streams. A custom stream could for example be "all by content + content with tags #foo or #bar + content in French language + content with images". All the calculation happens on write time, keeping reads super fast.

#devdiary

Jason Robinson

This was a good day. Not much in terms of code changes, but got rid of some shameful regexp in #Socialhome. This was used to find urls in content, for two purposes. To fetch OpenGraph / oEmbed and to linkify orphan text links.

It turned out pulling out urls from text isn't that trivial at all. Sure you can do a lot and that is what I did, and it worked great mostly. But then I realized the #Mozilla library Bleach, which I was using for cleaning HTML, has a pretty good url finder using the html5lib parser.

So, instead of regexp I ended up "abusing" bleach like this:

def find_urls_in_text(text):
    """Find url's from text.

    Bleach does the heavy lifting here by identifying the links.

    :param text: Text to search links from
    :returns: set of urls
    """
    urls = []

    def link_collector(attrs, new=False):
        href_key = (None, "href")
        urls.append(attrs.get(href_key))
        return None

    bleach.linkify(text, callbacks=[link_collector], parse_email=False,
                   skip_tags=["code"])
    return set(urls)

Normally, calling bleach.linkify will return you the same text with links going through additional processing. Instead, we're just using the callbacks it allows to define to capture the urls and then returning them in a set. A bit dirty I guess, but efficient and outsourcing to an established, performant, third-party library, instead of maintaining custom regexp.

#devdiary #python

Jason Robinson

Sigh, huge #UX problem with share replies on #Socialhome.. Damn you #diaspora protocol.. πŸ˜‘

Currently it's like this:

The first reply is on the original content. The rest of the replies are on the reshare I made with my diaspora account.

I'm really not sure how to go with this. Separate reply "sections" by share with replies might be the only way.. Then provide a "reply" button for each share "section"... It's kind of annoying, but otherwise the replies are not guaranteed to ever go to the right place in diaspora protocol network. Which would be even more annoying.

#devdiary

Jason Robinson

First share sent out from a #Socialhome node, yay 😁

Share stream visualization differences below. On #diaspora:

On Socialhome (in development still, not released:

The biggest single change is that shares are not shown separately in the stream. They (will) just have the effect of bumping content up to streams they would not otherwise be in. Still thinking how to represent remote comments on the share object, which we do store separately with its real guid. So, identifying comments targeting a share is not a problem, the problem is which parent to assign local replies to when sending out.

#devdiary

Jason Robinson

#Tests <3

Almost broke, again, all streams for anonymous users. So hard to forget they don't have a profile. Every time tests go kaboom.

Worth every (sometimes painful) minute.

#socialhome #devdiary

Jason Robinson

Also, implementing the same features in both the legacy (stable) #jQuery based stream and upcoming #VueJS stream really drives home how much better a framework like Vue is. Everything is just so much cleaner, efficient and generally easier to see from the code what is going on.

Thanks again @{Augier; augierle42e@diaspora-fr.org} for pushing the Vue frontend rewrite of #Socialhome πŸ‘Œ

#devdiary

Jason Robinson

is "Unshare" a good term for removing a share? πŸ€”

#socialhome #devdiary

Jason Robinson

Implementing "shares" for #Socialhome. This will be (at least on protocol level) to satisfy Diaspora "Reshare". It will be more like Twitter "Retweet" or Mastodon "Boost". The idea is that reshares will not be shown as separate items on the stream, but instead the reshared content will be "lifted" up to streams it would not otherwise be visible to. Optionally the user would be able to "quote" the share, adding their own note to it, which would make it a separate content item shown in streams even if pure shares were already shown.

How does this sound? Comments targeting the reshare from Diaspora will be tricky, but we will think of something πŸ€”

#devdiary

Jason Robinson

Profile picture update lands!

Even though we like seeing more and more #Django ponies in the stream, we recognize users might want a slightly more personal picture for their profile. So, finally, you can set your own profile picture! Changing the picture also updates it to your remote followers through the #federation layer.

Access the profile picture upload page through the drop-down in your profile. First upload an image, after which it is possible to set a center point for the automatic crop that happens. All profile images are square shape, but your uploaded image doesn't have to be.

This addition is available in the development branch and on https://socialhome.network which runs on the development branch.

#socialhome #changelog #devdiary

Socialhome HQ - Socialhome

Socialhome HQ

#Socialhome image uploads are now available on mobile browsers too. Previously it was only possibly to drag'n'drop an image into the editor. Now you can embed in place by using the new "camera" icon.

This change is in the development branch and will be in the next release. The official public server https://socialhome.network runs on the development branch.

Btw, profile picture upload is almost ready, coming soon!

#devdiary #changelog

Socialhome HQ - Socialhome

Socialhome HQ

Profile search lands in development branch

There is now a global #search in the right side of the header. The search returns matches for local and remote profiles based on their name and username part of the handle. Profiles marked with visibility "Self" or "Limited" are excluded from the search results. Profiles marked with visibility "Site" will be excluded if not logged in, leaving only public profile results. If a direct match happens with a full handle, a redirect is done directly to the searched profile. When searching for profiles based on handle, profile is fetched from remote if it isn’t found locally.

IMPORTANT for node maintainers. After pulling in this change, you MUST run the command python manage.py rebuild_index to create the search index. Not doing this will cause an error to be raised when trying to search. The indexes are kept up to date automatically after running this command once.

The future of search

In addition to profiles, search will be coming for tags and content too. There are some ideas how to represent the global search results page with these different kinds of items. It could be something like this:

In this #mockup, we break the results into their own result areas, making it easier to locate the result you were looking for. The profile and tags results would have pagination and the content grid below would scroll down, loading more results. This would make the search a kind of dynamic stream.

Opinions on this and searching social content in general welcome!

Where?

Test Socialhome at https://socialhome.network . This node follows the development branch and will always have the latest merged in features.

#socialhome #devdiary #changelog

Socialhome HQ - Socialhome

Socialhome HQ

Some basic #search on #Socialhome finally starting to come together. Only on my personal node atm though. Explains why many people got a follow from me just now as I'm eagerly searching and following people from my #diaspora contacts :) Remote direct match lookup is still to come though, so can't reach all pods who don't push over the relay system (btw, you should!). Currently the search is for profiles only. Tags will be added soon, after that full text for content.

On the technical side, after prototyping some solutions, went with django-haystack for the search framework and whoosh for the engine. Whoosh is a pure #Python backend with a file index. While this might not offer the performance we will need for large index full text search (from all content!), using Haystack on top means we can just switch transparently to #Elasticsearch at any time without touching pretty much any code - or even support using one or the other. Some people running nodes might not want to install ES.

Landing soon in master, you can try it at my own site if you are interested. It's basic but works.

#django #devdiary

Jason Robinson

Contact management lands

You can now access contacts you follow via the navbar "Contacts" menu item. This allows visiting contact profiles and unfollowing them. Check it out and let us know of any improvement ideas!

Next will add contact search. Will also add a "followers" contacts version at some point in the future.

About the following model

Socialhome following is done the way it works on #Twitter, ie the asymmetric model. This is why we call it "following" instead of "sharing" as in #Diaspora. There is no requirement of creating a social connection with someone to share with them. Following basically just means "I want to see this persons posts in my followed stream". It doesn't give the followed person any extra visibility to your posts or profile.

In the future once non-public content support lands (currently everything is "public"), there will be a way to manage aspect like groups for purely targeting content. Adding a person to this kind of list is separate from following. In other words, you don't need to follow someone to include them in a contact list. This is the "sharing" side that for example Diaspora has included in the "follow" side.

The contact management page will be refined as different kinds of contacts are available. Currently there are only "people I follow" and "people who follow me".

Where?

Give Socialhome a try at https://socialhome.network

#devdiary #changelog #socialhome

Socialhome HQ - Socialhome

Socialhome HQ