Micro Matt

dev

Banged my head against the wall for a bit on how to have dynamic blog routes on Write.as, so we can support custom redirects (as might be needed when moving from another platform to ours).

I dug into Go’s http library for a bit, then came across this Stack Overflow answer that pointed me in the right direction. Then it was a matter of how I’d make this multi-user and entirely dynamic, loading rules from the database for any blog that had redirect rules. For a minute, I was just going to hard-code things for the single blog that needs this feature to avoid more database overhead, but even that got messy, and it was better to do it right.

My solution was this: when the app starts up (and initializes all routes), query the database for all blogs with custom redirect rules. Then for each one, add a special handler that runs through all the custom rules for that blog. This handler only applies to the blog’s canonical URL (e.g. a custom domain) to keep things simple.

If a from path (the old URL we’re redirecting from) matches the address the visitor navigated to, move on to handling the request. There, we again match the from path to the current URL, and expand it into the desired to path (the destination) with any regex variables replaced — and finally redirect the user. This maintains support for regular expressions in these redirect rules, and most importantly, doesn’t add new overhead for other users or valid URLs. Phew, that was a challenge. But I’m pleased with the results!

#dev

Thoughts? Discuss...

Back to some coding today / tonight. Just finished the option to put posts on a sub-path, which mostly just lays the groundwork for things like static home pages (it won’t be widely available yet). Next I’m working on custom redirect support, which again won’t be widely available. Then it’ll be more work on our new category system.

Tonight I spent some time thinking about an easier post management UI, and ended up designing something that combines that with an early “drafts” system — that is, completely private posts tied to a blog, so you can preview them with your blog’s theme and share them with any team members before publishing. Basically, this will be an early, built-in version of Draft.as (without any version control).

It’s also very interesting to me, because it’ll live alongside our continued support for Anonymous posts, which are always useful as publicly-shareable “drafts.” In other words, each kind of post (anonymous, draft, blog post) will have a distinct use case, so we’re not adding pointless cruft or shoehorning any functionality into where it doesn’t fit. Excited to develop this, when time allows for it.

#dev

Thoughts? Discuss...

Shipping soon: native support for Markdown and HTML in blog descriptions!

This was a long-requested feature, and there’s really no reason not to support it — both Snap.as and our future Remark.as support it. Now you can do things like verify your blog for Mastodon and elsewhere in the IndieWeb.

Further reading: dev task T874, WriteFreely PR #531.

#dev #WriteFreely

Thoughts? Discuss...

Today I added a capability to Snap.as that’ll open up more customization options: the ability to mark an uploaded photo for a particular use. This way, for example, we can store a favicon for Write.as or an avatar for Remark.as, and Snap.as will keep only one image for that particular use.

#dev

Thoughts? Discuss...

Writing a Twitter thread introducing Remark.as and poking fun at “web3,” and I think I’m actually going to build some of these things I’m talking about. Really follow the joke to completion, and it’ll actually be interesting to have an open, social collection of “Neat Fun Things” in the product.

Last night I made more progress on the app. I’m trying not to get too bogged down by a checklist of “blog commenting” functionality, and focus more on the experience of “hanging out around blogs.” That means things will be a little funky when you arrive, and maybe a little disorienting — but that’s kind of the point. I don’t want to build a sterile, Facebook-ish environment.

As of last night, instead of landing in a network-wide space (the “Café”), I have the app landing on your “Buddy List.” So instead of overwhelming you with a feed-reader-style inbox of content to read, you’ll only see the people you care about (that you’re following), and a hint at when they posted last. From there, you can consciously choose when and where to engage.

#dev #comments #remarkAs

Thoughts? Discuss...

Continuing on some performance improvements after last week’s downtime. Today, I implemented some long-needed changes to reduce the number of UPDATEs happening on the database at any given moment.

Previously, every page load would immediately count the visitor and update the database. This worked perfectly fine when we were small, but now at normal traffic levels, and especially with spikes like the one we saw last week, this has become too much for the database to handle. Also since we use database replication, the issue has become visible to users, as (I believe) transactions pile up and things get out of sync between database servers.

With this change, many of those issues should go away. Some quick benchmarking showed that responses no longer pile up and gradually grind everything to a halt, as they would’ve before — even with high concurrency and sustained requests, the slowest response could be 600ms. In my tests, it seems now the application can handle at least three times as many concurrent visitors as it could before this change.

We generally see more traffic every day around 10-11am Eastern, so that should put this to the test tomorrow. But it’s looking good so far.

#dev #performance

Thoughts? Discuss...

Today I’ve been working on changes needed for closing Free account registrations at the end of the month. I’ve dragged my feet on this a bit, so decided to make the “close” date coincide with the new year.

The signup flow will change a bit. Marketing pages will start talking about the platform more in terms of everything it can do (with a paid account), instead of constantly mentioning how that differs from a free account. This change alone is making things so much easier from a marketing perspective — I’m finally seeing how much effort goes into catering to free users, and making sure they understand the limitations and differences.

Otherwise, I think we’ll let everyone use the “Invite” functionality to bring people in and give them free accounts, to make sure teachers and others aren’t left out of this transition.

#dev

Thoughts? Discuss...

One tricky thing: do we track which #categories are included in the post itself vs. in pure metadata?

Assuming all are inline, things are simple: on publish or update, we always parse hashtags in the body. If a tag isn’t there, we delete its association with the post. If it is there, we associate it with the post.

If we also want to have pure-metadata categories, so we don’t have to clutter up a post with visible hashtags, then we need to track which categories are inline vs. metadata-only, so we know which ones to remove on update and which ones to keep. (And now this is getting complicated.)

It’s mostly getting complicated in the UI. Assuming the editor now has a “categories” field, we’ll need to keep it in sync between inline / editor hashtags and metadata-only tags. I mean, it’s unlikely someone would use both; I don’t want to cater to that edge case, but I also don’t want to exclude it if we don’t have to.

We might also follow the pattern we’ve laid out with other add-on post metadata, like #authors. Right now, you can add an author in the Rich Text (RT) editor, but not the Plain Text (PT) editor. In this way, we keep our underlying flexibility, but the client / editing UI guides users toward the correct input method. I think we can assume that the PT editor is for focused writing and inline metadata; the RT editor is for exact control over presentation and metadata. I’m not sure if that’s entirely correct, but I think I have to start there.

#dev

Thoughts? Discuss...

Continuing yesterday’s work, internal support for #categories is finished. The result is basically a lightweight layer on top of the existing hashtag-based system.

Now whenever you create or modify a post, or move it to a blog, we’ll parse out the hashtags and automatically create categories from them, as necessary. Categories store original information about hashtags, plus a user-friendly title (which can include spaces, punctuation, capitalization you want, etc.) and a URL-friendly slug. (You can see some of the underlying code here.)

In this way, they’re completely optional and unobtrusive by default. If you care to carefully manage your categories, with this new system, you’ll be able to do it. If you just want to tag a post occasionally, this won’t slow you down. And if your needs change as you write more posts, this will be there when you decide to organize things.

I still have more testing to do before deploying this change, and even then, users won’t notice anything new yet. But the groundwork will be there for us to tackle the management side next.

#dev

Thoughts? Discuss...

Working on support for categories today. There are a few functional goals we’re trying to solve with this:

  • We can list all categories used on a blog
  • We can quickly filter all posts under a certain category
  • We can create tags with specific capitalization, Unicode characters, and spaces in the name
  • We can associate posts with certain categories via existing plain text tagging system

The last point in particular is pretty tricky to solve; all other points are solved easily by adding some new data structures. I’m thinking we’ll just store three pieces of data for each category: a slug (e.g. united-states), a title (e.g. United States), and a normalized “lookup slug” that can be represented by a hashtag (e.g. unitedstates).

Then we’ll do some magic on the backend when creating or updating a post that parses the post and creates a new category automatically and / or associates the post with an existing category. That will allow existing posts to use this new categorization system. Then we might also support a new “silent” way of adding categories via a new API field, so you can associate a post with a category without inserting it into the body of the post.

Just some implementation ideas so far; we’ll see if this works in practice.

#dev #categories

Thoughts? Discuss...