#devdiary

Making #Socialhome pre-cache streams content id's in #Redis for each user. Going to have super fast stream loading 😍 Basically something like this:

  • On save -> push to a Redis ordered set for each user for each stream the content ID
  • On stream load, pull out X items from the ordered set in reverse order
  • If we have enough items -> fetch objects from DB and render
  • If we need more items, collect enough using normal queryset filters (slow, current method) -> fetch objects from DB and render

I tried using a Redis List at first, but I couldn't figure out how to make "load more" work with that. If the client says "I want all after item X" then I have to pull out X items from the Redis list after that item. But ordered sets are perfect since they have unique members and items can be neatly pulled out in order from either direction.

Actually the Socialhome streams are not even slow at the moment. But doing this pre-caching allows for any kind of complex calculations to be made for custom streams. A custom stream could for example be "all by content + content with tags #foo or #bar + content in French language + content with images". All the calculation happens on write time, keeping reads super fast.

#devdiary

Jason Robinson

This was a good day. Not much in terms of code changes, but got rid of some shameful regexp in #Socialhome. This was used to find urls in content, for two purposes. To fetch OpenGraph / oEmbed and to linkify orphan text links.

It turned out pulling out urls from text isn't that trivial at all. Sure you can do a lot and that is what I did, and it worked great mostly. But then I realized the #Mozilla library Bleach, which I was using for cleaning HTML, has a pretty good url finder using the html5lib parser.

So, instead of regexp I ended up "abusing" bleach like this:

def find_urls_in_text(text):
    """Find url's from text.

    Bleach does the heavy lifting here by identifying the links.

    :param text: Text to search links from
    :returns: set of urls
    """
    urls = []

    def link_collector(attrs, new=False):
        href_key = (None, "href")
        urls.append(attrs.get(href_key))
        return None

    bleach.linkify(text, callbacks=[link_collector], parse_email=False,
                   skip_tags=["code"])
    return set(urls)

Normally, calling bleach.linkify will return you the same text with links going through additional processing. Instead, we're just using the callbacks it allows to define to capture the urls and then returning them in a set. A bit dirty I guess, but efficient and outsourcing to an established, performant, third-party library, instead of maintaining custom regexp.

#devdiary #python

Jason Robinson

Sigh, huge #UX problem with share replies on #Socialhome.. Damn you #diaspora protocol.. πŸ˜‘

Currently it's like this:

The first reply is on the original content. The rest of the replies are on the reshare I made with my diaspora account.

I'm really not sure how to go with this. Separate reply "sections" by share with replies might be the only way.. Then provide a "reply" button for each share "section"... It's kind of annoying, but otherwise the replies are not guaranteed to ever go to the right place in diaspora protocol network. Which would be even more annoying.

#devdiary

Jason Robinson

First share sent out from a #Socialhome node, yay 😁

Share stream visualization differences below. On #diaspora:

On Socialhome (in development still, not released:

The biggest single change is that shares are not shown separately in the stream. They (will) just have the effect of bumping content up to streams they would not otherwise be in. Still thinking how to represent remote comments on the share object, which we do store separately with its real guid. So, identifying comments targeting a share is not a problem, the problem is which parent to assign local replies to when sending out.

#devdiary

Jason Robinson

#Tests <3

Almost broke, again, all streams for anonymous users. So hard to forget they don't have a profile. Every time tests go kaboom.

Worth every (sometimes painful) minute.

#socialhome #devdiary

Jason Robinson

Also, implementing the same features in both the legacy (stable) #jQuery based stream and upcoming #VueJS stream really drives home how much better a framework like Vue is. Everything is just so much cleaner, efficient and generally easier to see from the code what is going on.

Thanks again @{Augier; augierle42e@diaspora-fr.org} for pushing the Vue frontend rewrite of #Socialhome πŸ‘Œ

#devdiary

Jason Robinson

is "Unshare" a good term for removing a share? πŸ€”

#socialhome #devdiary

Jason Robinson

Implementing "shares" for #Socialhome. This will be (at least on protocol level) to satisfy Diaspora "Reshare". It will be more like Twitter "Retweet" or Mastodon "Boost". The idea is that reshares will not be shown as separate items on the stream, but instead the reshared content will be "lifted" up to streams it would not otherwise be visible to. Optionally the user would be able to "quote" the share, adding their own note to it, which would make it a separate content item shown in streams even if pure shares were already shown.

How does this sound? Comments targeting the reshare from Diaspora will be tricky, but we will think of something πŸ€”

#devdiary

Jason Robinson

Profile picture update lands!

Even though we like seeing more and more #Django ponies in the stream, we recognize users might want a slightly more personal picture for their profile. So, finally, you can set your own profile picture! Changing the picture also updates it to your remote followers through the #federation layer.

Access the profile picture upload page through the drop-down in your profile. First upload an image, after which it is possible to set a center point for the automatic crop that happens. All profile images are square shape, but your uploaded image doesn't have to be.

This addition is available in the development branch and on https://socialhome.network which runs on the development branch.

#socialhome #changelog #devdiary

Socialhome HQ - Socialhome

Socialhome HQ

#Socialhome image uploads are now available on mobile browsers too. Previously it was only possibly to drag'n'drop an image into the editor. Now you can embed in place by using the new "camera" icon.

This change is in the development branch and will be in the next release. The official public server https://socialhome.network runs on the development branch.

Btw, profile picture upload is almost ready, coming soon!

#devdiary #changelog

Socialhome HQ - Socialhome

Socialhome HQ

Profile search lands in development branch

There is now a global #search in the right side of the header. The search returns matches for local and remote profiles based on their name and username part of the handle. Profiles marked with visibility "Self" or "Limited" are excluded from the search results. Profiles marked with visibility "Site" will be excluded if not logged in, leaving only public profile results. If a direct match happens with a full handle, a redirect is done directly to the searched profile. When searching for profiles based on handle, profile is fetched from remote if it isn’t found locally.

IMPORTANT for node maintainers. After pulling in this change, you MUST run the command python manage.py rebuild_index to create the search index. Not doing this will cause an error to be raised when trying to search. The indexes are kept up to date automatically after running this command once.

The future of search

In addition to profiles, search will be coming for tags and content too. There are some ideas how to represent the global search results page with these different kinds of items. It could be something like this:

In this #mockup, we break the results into their own result areas, making it easier to locate the result you were looking for. The profile and tags results would have pagination and the content grid below would scroll down, loading more results. This would make the search a kind of dynamic stream.

Opinions on this and searching social content in general welcome!

Where?

Test Socialhome at https://socialhome.network . This node follows the development branch and will always have the latest merged in features.

#socialhome #devdiary #changelog

Socialhome HQ - Socialhome

Socialhome HQ

Some basic #search on #Socialhome finally starting to come together. Only on my personal node atm though. Explains why many people got a follow from me just now as I'm eagerly searching and following people from my #diaspora contacts :) Remote direct match lookup is still to come though, so can't reach all pods who don't push over the relay system (btw, you should!). Currently the search is for profiles only. Tags will be added soon, after that full text for content.

On the technical side, after prototyping some solutions, went with django-haystack for the search framework and whoosh for the engine. Whoosh is a pure #Python backend with a file index. While this might not offer the performance we will need for large index full text search (from all content!), using Haystack on top means we can just switch transparently to #Elasticsearch at any time without touching pretty much any code - or even support using one or the other. Some people running nodes might not want to install ES.

Landing soon in master, you can try it at my own site if you are interested. It's basic but works.

#django #devdiary

Jason Robinson

Contact management lands

You can now access contacts you follow via the navbar "Contacts" menu item. This allows visiting contact profiles and unfollowing them. Check it out and let us know of any improvement ideas!

Next will add contact search. Will also add a "followers" contacts version at some point in the future.

About the following model

Socialhome following is done the way it works on #Twitter, ie the asymmetric model. This is why we call it "following" instead of "sharing" as in #Diaspora. There is no requirement of creating a social connection with someone to share with them. Following basically just means "I want to see this persons posts in my followed stream". It doesn't give the followed person any extra visibility to your posts or profile.

In the future once non-public content support lands (currently everything is "public"), there will be a way to manage aspect like groups for purely targeting content. Adding a person to this kind of list is separate from following. In other words, you don't need to follow someone to include them in a contact list. This is the "sharing" side that for example Diaspora has included in the "follow" side.

The contact management page will be refined as different kinds of contacts are available. Currently there are only "people I follow" and "people who follow me".

Where?

Give Socialhome a try at https://socialhome.network

#devdiary #changelog #socialhome

Socialhome HQ - Socialhome

Socialhome HQ

Followed stream is here!

Finally! The most basic feature of them all, seeing what the people you're interested in are writing about, has finally landed in #Socialhome. Why did it take this long? Well, before this many things had to be implemented, for example contacts on the #federation layer and actually following people in the UI.

Additionally, added a preferences screen for users. There is one preference available at the moment, choice of landing page. When logged in, users used to always see their own profile. Now you can choose what page you want to see when you log in or click the navbar logo. Tip: if you change it to a stream, access your profile via the "My Profile" link in the navbar.

Currently "profile" is the default (to keep old behaviour). I'm thinking of making the "followed" stream the default (which mimics what you see on other networks normally). Opinions?

Next: contact management! Have a feature that would make Socialhome more useful to you or make you want to try it? Let me know!

#devdiary #changelog

Socialhome HQ

Following (top) and follower (bottom) counts now in the profile panel, for local profiles.

For self, these will be soon links to contact management pages. When viewing other profiles, they are just text, since we don't want to show other people who is following who.

Opinion: should there be a user setting to hide these counts?

Next: Contact management and stream of contacts content.

#socialhome #devdiary #changelog

Socialhome HQ

Profile content stream + minor redesign

Added now a new "All content" stream for each profile. This can be found in the user profile. It is always rendered by default if the profile is remote or is a local profile with no pinned content.

Also related to this a minor redesign of the profile page. The "stamped" area with profile information is now centered with two columns on xl-large screens, two columns on left for large screens and full-width on medium or smaller. It now has a user actions drop-down for modifying settings or profile content, and for other user profiles there are the home/follow/unfollow buttons as in stream content.

The navbar also received a rewrite as bumping a #Bootstrap 4 alpha version broke things again.

Feedback

Would love to receive feedback on the usefulness of the profile stream, or streams in #Socialhome in general. If you have tried Socialhome, did you find the grid layout useful or confusing?

It is likely the design will live quite a bit in the future. The current one was really a "let's try this" idea which still needs to be proven. Underneath the code is starting to suffer from "spaghetti code" problems with lots of #jQuery events firing to pull the strings. I'm seriously considering rewriting the stream with either #React or maybe #Vue. Before that happens, it would be nice to have a clearer idea of the future stream #UX.

If you want to give feedback or discuss these things, feel free to leave some comments here or get in touch via chats or GitHub issues. Private messages don't yet work in the #federation layer so please don't send those yet.

Project info

As a reminder for new readers. Socialhome is a #Django powered project aiming to create a platform that allows users to create a simple social profile with dynamic content. All content is federated using the #Diaspora protocol. Streams are in a central role, and we plan to have many types of streams, including custom per user streams.

You can find the code here and some documentation. The official site is https://socialhome.network which is a Socialhome instance itself and open for registration. Feel free to play around and feedback is always welcome!

#devdiary #changelog

Socialhome HQ - Socialhome

Socialhome HQ