#socialhome

Sunday project: Huginn, Docker & Ansible

Some time ago I set up a #Twitter account posting #WWII "as it happened" events to #Socialhome (see here). This was accomplished using Zapier that reads the tweets and then POST's to the Socialhome API.

All that worked great and #Zapier is really nice. Except it's limited to 100 events per month on the free tier and this seems wasn't enough for the WWII tweets coming in. Zapier paid plans are from $20/mo which is way too much just to replicate some tweets.

At first I thought about making my own app to do all this but then I remembered running into Huginn, which I have been wanting to install anyway. It's basically a self-hosted #IFTT or Zapier, but allowing for even more complex stuff like unlimited chaining of events.

Docker + Ansible

I'm pretty new to #Docker but eventually I want to dockerize more of my apps. So of course I was happy to see #Huginn has a Docker image, allowing me to not have to muck about with any more #RoR apps. I use #Ansible for pretty much everything, so it made sense to build a playbook to deploy the app in a Docker container.

Luckily (or thanks to the fantastic Ansible team that is!), maintaining Docker containers with Ansible is as easy as doing anything else with it.

I chose to keep the web server out of the containers to use my standard #LetsEncrypt role for getting SSL on top. This made the role very simple, having only to 1) fetch SSL cert, 2) set up Apache to proxy and 3) push up a container.

The docker_container Ansible module call is basically just this:

 ---
 - docker_container:
   name: huginn
   image: huginn/huginn
   volumes:
     - "{{ huginn_db_volume_mount }}:/var/lib/mysql"
   env:
     APP_SECRET_TOKEN: "{{ huginn_secret }}"
     DOMAIN: "{{ huginn_domain }}"
     FORCE_SSL: "{{ huginn_https}}"
     INVITATION_CODE: "{{ huginn_invitation_code }}"
     MAIL_FROM_ADDRESS: "{{ huginn_mail_from }}"
     RAILS_ENV: production
     SMTP_DOMAIN: "{{ huginn_smtp_domain }}"
     SMTP_PASSWORD: "{{ huginn_smtp_password }}"
     SMTP_PORT: "{{ huginn_smtp_port }}"
     SMTP_SERVER: "{{ huginn_smtp_server }}"
     SMTP_USER_NAME: "{{ huginn_smtp_username }}"
     TWITTER_OAUTH_KEY: "{{ huginn_twitter_key }}"
     TWITTER_OAUTH_SECRET: "{{ huginn_twitter_secret }}"
   register: _docker_huginn

The container IP is then picked up from _docker_huginn by Apache to do the proxying.

The playbook itself, without any extra customization is pretty much here:

 ---
 - hosts: myhost
   become: yes
   become_user: root

   vars:
     letsencrypt_email: youremail@example.com
     huginn_domain: huginn.example.com
     huginn_invitation_code: changeme!
     huginn_secret: changeme!

   roles:
     - huginn

The result is the ansible-huginn role. Tested on #Ubuntu 14.04. Right now a bit limited in terms of configuration that can be passed in. Will expand as necessary.

Fun Huginn stuff

Some initial things I want to do is to use the chaining to make the tweet replicating not embed the tweet, but instead rip out the text + media and create a normal post instead. That should make for cleaner streams with less Twitter JS firing off.

Any tips on fun stuff to do with Huginn?

huginn/huginn
huginn - Create agents that monitor and act on your behalf. Your agents are standing by!

Jason Robinson

Socialhome client shcli now available as a Snap

shcli allows creating content using either the command line or #Python API. Now in addition to installing from PyPi, it's available as a #Snap.

If on a recent #Ubuntu desktop or server, just do sudo snap install shcli and start posting. On the desktop you can also use the software manager to install it as any other package.

On other systems, for example #Arch, #Debian, #Gentoo, #Fedora, #OpenSUSE, #OpenEmbedded, #Yocto, #OpenWRT and #Solus, install the Snap daemon first.

#socialhome

Socialhome HQ

Testing #Socialhome #Python client shcli #snap package. Hello, world! This time in strict confinement :P

Jason Robinson

Testing #Socialhome #Python client shcli #snap package. Hello, world!

Jason Robinson

Any recommendations for web based translation tools for #opensource projects? Need to start thinking of translations for #Socialhome.

Jason Robinson

Starting to unfollow people from this account, following instead from my #Socialhome account jaywink@jasonrobinson.me .. please if you want to see my posts, follow that account instead. I have not been posting from this account for a while.

Socialhome doesn't support private posts yet, so if you want to hit me up with those, please mention account.

Btw, for those on iliketoast.net. The pod will keep running until a migration process has been in place for some time, maybe even longer. I will update it to 0.7.0.0 in the coming days, apologies for taking such a long time.

I Like Toast

Jason Robinson

De protocolos y alianzas en las redes sociales libres

À propos des protocoles et des alliances des réseaux sociaux libres

About protocols and alliances in free social networks

ES: Quién "habla" qué protocolo y quién está federado con quién. FR: Qui "parle" quel protocole et qui est fédéré avec qui. EN: Who's "speaking" what protocols and who's federated with whom.

Source: Mike Macgirvin in RedMatrix https://macgirvin.com/channel/mike/?f=&mid=a167ea295beeca0389a34a97c94c987765375a31fc49a090d8602278dbc319fc@macgirvin.com

#protocolo #protocolos #protocol #protocols #protocole #protocoles #internet #redessociales #redes-sociales #redsocial #red-social #socialmedia #social-media #social-networks #social-network #socialnetworks #socialnetwork #réseaux-sociaux #reseaux-sociaux #réseauxsociaux #reseauxsociaux #réseau-social #réseausocial #reseausocial #fediverse #federación #lafederación #thefederation #the-federation #federation #fédération #la-fédération #lafédération #GNUSocial #GNU-Social #Pleroma #postActiv #Quitter #Mastodon #Friendica #Hubzilla #RedMatrix #Diaspora #SocialHome #GangGo #NextCloud #IndieWeb #MediaGoblin #Pumpio #Pump-io

Miguel

Making #Socialhome pre-cache streams content id's in #Redis for each user. Going to have super fast stream loading 😍 Basically something like this:

  • On save -> push to a Redis ordered set for each user for each stream the content ID
  • On stream load, pull out X items from the ordered set in reverse order
  • If we have enough items -> fetch objects from DB and render
  • If we need more items, collect enough using normal queryset filters (slow, current method) -> fetch objects from DB and render

I tried using a Redis List at first, but I couldn't figure out how to make "load more" work with that. If the client says "I want all after item X" then I have to pull out X items from the Redis list after that item. But ordered sets are perfect since they have unique members and items can be neatly pulled out in order from either direction.

Actually the Socialhome streams are not even slow at the moment. But doing this pre-caching allows for any kind of complex calculations to be made for custom streams. A custom stream could for example be "all by content + content with tags #foo or #bar + content in French language + content with images". All the calculation happens on write time, keeping reads super fast.

#devdiary

Jason Robinson

One stop shop idea

I've had this idea for awhile. I'm considering starting a GoFundMe for raising money towards building a powerful server that will host an instance of every #Foss Social Network including, but not limited to, #diaspora, #libertree, #hubzilla, and #socialhome.

Before I face that mountain, I would like some feedback on it. We're switching to a fiber optic connection so my speeds will shoot up tremendously. I have enough equipment to handle the bandwidth and heavy loads, I just need a server to handle all of it.

Any thoughts?

#FreeSoftware #SoftwareLibre

Theatre

This was a good day. Not much in terms of code changes, but got rid of some shameful regexp in #Socialhome. This was used to find urls in content, for two purposes. To fetch OpenGraph / oEmbed and to linkify orphan text links.

It turned out pulling out urls from text isn't that trivial at all. Sure you can do a lot and that is what I did, and it worked great mostly. But then I realized the #Mozilla library Bleach, which I was using for cleaning HTML, has a pretty good url finder using the html5lib parser.

So, instead of regexp I ended up "abusing" bleach like this:

def find_urls_in_text(text):
    """Find url's from text.

    Bleach does the heavy lifting here by identifying the links.

    :param text: Text to search links from
    :returns: set of urls
    """
    urls = []

    def link_collector(attrs, new=False):
        href_key = (None, "href")
        urls.append(attrs.get(href_key))
        return None

    bleach.linkify(text, callbacks=[link_collector], parse_email=False,
                   skip_tags=["code"])
    return set(urls)

Normally, calling bleach.linkify will return you the same text with links going through additional processing. Instead, we're just using the callbacks it allows to define to capture the urls and then returning them in a set. A bit dirty I guess, but efficient and outsourcing to an established, performant, third-party library, instead of maintaining custom regexp.

#devdiary #python

Jason Robinson

Awesome, this library will allow me to add #emoji reactions to #Socialhome that can be #federated out as a standard like, if positive 👍 😁

https://github.com/wooorm/emoji-emotion

wooorm/emoji-emotion
emoji-emotion - List of emoji rated for valence in JSON

Jason Robinson

#Socialhome v0.4.0 released

The focus of work lately has been implementing missing features for proper daily usage. One of these is allowing users to share content created by other people. This is now partially possible. Remote shares are now processed and users are able to create shares, but shares are not yet themselves taken into account when creating a stream. Will write another post related to the status with shares and how they will function.

Check out the release here: https://github.com/jaywink/socialhome/releases/tag/v0.4.0

Try it!

Registrations are open at https://socialhome.network . Please be aware some features are still missing, but otherwise the software is very stable. Let us know what you think if you decide to try it!

Participate

Are you a #Django, #Python or #VueJS coder? Want to build a social network? Check out the contribution guide and get in touch!

Changelog

Update notes

This release contains long running migrations. Please allow up to 10 minutes for the migrations to run, depending on your database size.

Added

  • Allow user to change profile picture. (#151)

    Profile menu now has an extra option "Change picture". This allows uploading a new picture and optionally setting focus point for cropping a picture that is not square shape.

  • Federate local profiles to remote followers on save. (#168)

  • Process remote profiles entities on receive.

    Remote profiles were so far only created on first encounter. Now we also process incoming Profile entities from the federation layer.

  • When following a remote profile, federate profile to them at the same time.

  • It is now possible to expose statistics from a Socialhome node. This includes counts for users (total, 30 day, 6 month), local content and local replies. These will be exposed via the NodeInfo documents that for example the-federation.info node list consumes.

    By default statistics is off. Admins can switch the counts on by setting environment variable SOCIALHOME_STATISTICS=True and restarting Socialhome.

  • Add user API token view. Allows retrieving an API token for usage in clients and tools. Allows also regenerating the token if it has been lost or exposed.

  • Added bookmarklet to easily share external pages. The bookmarklet can be bookmarked from the 'Create' page. (#138)

    Sharing with the bookmarklet will copy the page url, title and optionally selected text into the create content text area. The bookmarklet is compatible with Diaspora, so for example the Firefox sharing service will work.

  • Support receiving 'Share' entities. Show amount of shares on content. (#206)

  • Show replies to shares on the original shared content. (#206)

  • Add share endpoint to Content API. This enables creating and removing shares via the API. (#206)

  • Allow sharing content. Clicking the share counter icon exposes a 'Share' button which when clicked will create a share. (#206)

  • Allow unsharing content. Clicking the share counter icon exposes an 'Unshare' button (assuming the user has shared the content) which when clicked will remove the share. (#206)

  • Federate local shares to remote nodes. (#206)

  • There is now a 'My content' stream link in the navbar 'Streams' dropdown. This goes to your own profile all content stream.

  • Add user preference for the new stream refactoring. If enabled, all streams that have a new version in progress will be rendered with the new frontend code based on Vue.js. (#202)

    Warning! The new frontent code doesn't have all the features of the current on yet.

  • Content API has three new read only fields available:

    • local, boolean whether the content is local or remote.
    • reply_count, count of replies (including replies on shares)
    • shares_count, count of shares
  • Make email notifications nicer by using HTML templates in addition to the plain text version. (#206)

    In addition to reply and follow notifications, send also when own content is shared.

Changed

  • Breaking change. Content API results now return visibility as a string ('public', 'limited', 'site' or 'self'), not an integer.

Fixed

  • There was no notification sent out when a local user followed a local user. This has now been fixed.

Removed

  • Breaking change. Removed Content, Profile and Users API LIST routes. For now these are seen as not required for building a client and allow unnecessarily easy data mining.

  • Removed content modal. Clicking timestamp in grid now directly loads the content detail view. (#162)

    Loading the content in a modal was an early experiment and didn't end out very usable.

  • Removed reply button from replies. Technically, threaded replies are possible but the UI implementation is not done. Replying to a reply will be back once UI and federation layer will handle threaded replies properly.

#thefederation #federation

jaywink/socialhome
socialhome - A federated social home.

Socialhome HQ

Sigh, huge #UX problem with share replies on #Socialhome.. Damn you #diaspora protocol.. 😑

Currently it's like this:

The first reply is on the original content. The rest of the replies are on the reshare I made with my diaspora account.

I'm really not sure how to go with this. Separate reply "sections" by share with replies might be the only way.. Then provide a "reply" button for each share "section"... It's kind of annoying, but otherwise the replies are not guaranteed to ever go to the right place in diaspora protocol network. Which would be even more annoying.

#devdiary

Jason Robinson

https://twitter.com/jaywink/status/901825605381103616

#Zapier webhook to #Socialhome #API test


Jason Robinson

First share sent out from a #Socialhome node, yay 😁

Share stream visualization differences below. On #diaspora:

On Socialhome (in development still, not released:

The biggest single change is that shares are not shown separately in the stream. They (will) just have the effect of bumping content up to streams they would not otherwise be in. Still thinking how to represent remote comments on the share object, which we do store separately with its real guid. So, identifying comments targeting a share is not a problem, the problem is which parent to assign local replies to when sending out.

#devdiary

Jason Robinson

#Tests <3

Almost broke, again, all streams for anonymous users. So hard to forget they don't have a profile. Every time tests go kaboom.

Worth every (sometimes painful) minute.

#socialhome #devdiary

Jason Robinson

Also, implementing the same features in both the legacy (stable) #jQuery based stream and upcoming #VueJS stream really drives home how much better a framework like Vue is. Everything is just so much cleaner, efficient and generally easier to see from the code what is going on.

Thanks again @{Augier; augierle42e@diaspora-fr.org} for pushing the Vue frontend rewrite of #Socialhome 👌

#devdiary

Jason Robinson

is "Unshare" a good term for removing a share? 🤔

#socialhome #devdiary

Jason Robinson

https://socialhome.network/ это агрегатор соц площадок? #lang_ru #socialhome

Socialhome HQ - Socialhome

satorkain@diaspora.zone

Finally squashed the stats peaks on The-Federation.info.

Now, if only the number of active users would go up instead of declining since the peak 2,5 years ago :(

#diaspora #friendica #hubzilla #socialhome #ganggo #thefederation

Jason Robinson

Implementing "shares" for #Socialhome. This will be (at least on protocol level) to satisfy Diaspora "Reshare". It will be more like Twitter "Retweet" or Mastodon "Boost". The idea is that reshares will not be shown as separate items on the stream, but instead the reshared content will be "lifted" up to streams it would not otherwise be visible to. Optionally the user would be able to "quote" the share, adding their own note to it, which would make it a separate content item shown in streams even if pure shares were already shown.

How does this sound? Comments targeting the reshare from Diaspora will be tricky, but we will think of something 🤔

#devdiary

Jason Robinson

The day is getting closer when I've got enough functionality in #Socialhome to be able to fully replace my #Diaspora account. I'll be posting less from this account from now on. If you like what I share, please throw a follow at jaywink@jasonrobinson.me (profile link).

Limited posts and mentions are not yet supported on Socialhome, so if you want to mention me - use this account still for a while! :P

Ps. For users on iliketoast.net - don't worry! The pod isn't going anywhere. Not at least until sometime in the future migration has been in place for a while so that users can safely migrate to other pods 👍

Jason Robinson

How to: creating content via shcli

Socialhome has an API which covers most of the use cases for building a client. There is also a Python library/cli client that allows creating content over the API. This is a short introductory on how to use that.

shcli requires Python 3.4+. The client has only been tested on Linux, though it should work on Windows and Mac too. The only real dependency it has is the requests library.

Create a Python environment

Either install shcli in a virtualenv or globally, depending on your use case. For desktop users, a global installation makes sense.

Virtualenv

Install virtualenv for your system and create a Python 3 virtualenv and activate it.

Global install

Ensure you have a compatible Python 3 version of pip installed. For example on Ubuntu, this would be the python3-pip package.

Install shcli

Virtualenv:

 pip install shcli

Globally (for example on Ubuntu):

 sudo pip3 install shcli

Phew, now we're past the hard part.

Get API token

Go to your Socialhome account profile and find API token from the profile menu. Copy the token for later use. If you ever lose it, this is the place to generate a new token.

Post stuff!

Using the tool is easy. It currently only allows creating content, so commands and options are not plenty. More will be added in the future.

The client takes a command, a domain, a token and then parameters for text and visibility. The domain is your Socialhome account domain (for example socialhome.network) and token the API token copied in the previous step. To protect the token, the client always uses a HTTPS connection. This is why the protocol is not given in the command.

An example:

 shcli create socialhome.network 1234567890abcdefg \
     -t 'Hello world!' -v public

For complex creations, the client can retrieve the text from a file. Instead of -t, pass in the file name with the -f parameter. Visibility can be one of public, limited, self or site.

That's it pretty much! The response will be either the created content as JSON or some kind of error, if something went wrong.

Python API

You can also use the API through Python by importing shcli.create and passing in the same arguments as for the cli create command.

Code

The repo for shcli is on GitHub licensed under MIT.

#socialhome #howto #python #api

Socialhome HQ

The Federation

The Federation refers to a global social network composed by nodes talking together. Each of them is an installation of a software which supports the #diaspora* protocol.

The site

Visit The-Federation.info for node lists, statistics and information how to list your own node.

Follow

This account will be posting out statistics on a daily/weekly basis. The posts will be tagged according to frequency, so instead of following the account you can also subscribe to the hashtags. At least the following hashtags will be used:

#thefederation #diaspora #friendica #hubzilla #ganggo #socialhome

The Federation

Hello, World! From #Socialhome CLI client.

Jason Robinson

First part of upcoming #VueJS rewrite of #Socialhome streams merged in. Thanks, christophehenry!

Down with #jQuery, bring up the modern JS stack!

Socialhome HQ

Socialhome.Network

♲ mar_k83@mastodon.social:

https://mastodon.social/media/lvLWwqZ1xZiyUtr_Xn4 New network in the Föderation = socialhome.network #socialhome #network #social #facebook
https://files.mastodon.social/media_attachments/files/001/169/230/original/b3bbf5139bdfbb27.png

[l]

Delete

Socialhome is now on The-Federation.Info

You can now find and track #Socialhome nodes on https://the-federation.info 👍

On a related note. We don't have a logo. Anyone want to contribute one? Fame and fortune will be yours! <3

the federation - a statistics hub
Podlist and statistics for The Federation (diaspora*, Friendica, Hubzilla).

Socialhome HQ