The Small Website Discoverability Crisis (2021)

2023-11-1514:31557253www.marginalia.nu

There are a lot of small websites on the Internet: Interesting websites, beautiful websites, unique websites. Unfortunately they are incredibly hard to find. You cannot find them on Google or Reddit,…

There are a lot of small websites on the Internet: Interesting websites, beautiful websites, unique websites.

Unfortunately they are incredibly hard to find. You cannot find them on Google or Reddit, and while you can stumble onto them with my search engine, it is not in a very directed fashion.

It is an unfortunate state of affairs. Even if you do not particularly care for becoming the next big thing, it’s still discouraging to put work into a website and get next to no traffic beyond the usual bots.

You get a dead-sea effect. Traffic is evaporating, and small websites are dying, which brings even fewer visitors. Rinse and repeat.

Blogs limp along through RSS and Atom, but relying on feeds shapes everything you write into a blog entry. It’s stifling, homogenizing. The blogosphere, what remains of it, is incredibly samey.

I feel there ought to be a solution to this, a better way of doing things that can help, and perhaps the Internet as a whole is an irredeemable mess that will never mend, but maybe we can (somehow) make it easier for those who are actually looking to find what they seek.

Maybe there are lessons that can be drawn from what works on Gemini, and what doesn’t work on HTTP, that can synthesize into a sketch for a solution.

Gemini seems to be discovering automatic link feeds (e.g. Antenna), and on gemini-scale it works pretty well. But I’m just going to state that automatic link feeds do not seem to work on HTTP any more. You end up with a flood of astroturfing, vapid click-bait and blogspam (i.e. reddit). Stemming the flood demands a ton of moderation and still results in dismal results.

As a whole, I think centralized and algorithmic approaches are extremely exposed to manipulation when applied on the internet.

Web rings are cute, but I think they are a bit too random to help. Likewise, curated link directories were a thing back when the Internet was in its infancy, but the task of maintaining such a directory is a full time job.

You could go for some sort of web-of-trust model to only allow trusted submitters access to an automatic link feed, but that practice is excluding and creates yet more walled gardens, which impairs the very discoverability I’m trying to help.

Instead, perhaps there is a much simpler solution.

A proposal, dear reader: Create a list of bookmarks linking to websites you find interesting, and publish it for the world to see. You decide what constitutes “interesting”.

The model is as recursive as it is simple. There is nothing preventing a list of bookmarks from linking to another list of bookmarks.

The creation of a bookmark list is a surprisingly fun project, it has some of the appeal of scrapbooking; and the end-result is also appealing to browse through.

It’s a bit strange, almost nobody seems to be doing this. Looking through a sample of personal websites, very few of them has links to other personal websites. A hyperlink isn’t a marriage proposal. It is enough to find some redeeming quality in a website to link to it. It costs nothing, and helps bring traffic to pages that you yourself think deserve it.

If we actually want these small websites to flourish as a healthy community, we need to promote each other much more than we do. It is advertisement, yes, but in earnest. I like it when other people link to my stuff. What sort of hypocrite would I then be if I only ever linked to my own websites?

Leading by example, I set up my own list of bookmarks:


Read the original article

Comments

  • By kyledrake 2023-11-1515:4912 reply

    Neocities (disclosure: I work on it) has taken steps to try to improve small personal web site discoverability, which ends up being like a platform for people making web sites with a hybrid social component https://neocities.org

    I like the idea of calling this the small web, I usually go with something like "personal web site" or "home pages" but it's never quite stuck for me. I hope they've added Neocities to the Kagi small web search because there's some pretty incredible sites available for that and our compiled sitemap will make importing easy: https://neocities.org/browse

    The framing for this stuff is usually something like "wow remember the crazy 90s web" nostalgia pieces or "this is an active resistance against Facebook come join us in the lonely space nobody goes to." But really there's some incredible, magical content that requires the canvas the web provides, that isn't on the social media super-platforms and people very much still use the web to access them. Neocities alone serves hundreds of millions of views per month across all the sites, there's still a lot of web surfing going on.

    I would actually argue that having a web site gives you more exposure for your content than an average social media account, because sans a few lucky accounts, most are being throttled and limited by weird algorithms to prevent people from seeing your content organically. Your google search ranking might not be great, but people share links all over the place, including in private channels (think Slack/Discord/IRC/IMs) and you can still get meaningful distribution of your content this way.

    To paraphrase @izs "if you build it, they will come", is a misquote from a Kevin Costner movie about baseball ghosts, but if you build a good site with good content, people do just magically show up through mechanisms I don't myself quite understand yet. It's pretty cool to see new sites on Neocities that are unusually interesting and know they'll organically get view counts into the millions before it happens.

    • By hinkley 2023-11-1516:472 reply

      Re: Field of Dreams

      If you look at this story from anyone else’s perspective, right up until the last few moments this is a story about a man with untreated schizophrenia or temporal lobe seizures escalating his illness to the point of kidnapping someone and transporting them across state lines.

      Almost every company in the dot com boom was convinced the headlights at the end of their story would be vindication, not the ambulance coming to take them to a psychiatric ward. Almost all of them were wrong.

      • By zpeti 2023-11-1518:382 reply

        My mentor who inspired me to be an entrepreneur was diagnosed with schizophrenia and is basically spending the rest of his life in hospital…

        I really think there’s something in schizophrenia beyond the illness. A lot People with it normally get symptoms around 27-28 but achieve insane amounts before then (same as my mentor)

        • By nwiswell 2023-11-1519:38

          > I really think there’s something in schizophrenia beyond the illness.

          I don't think there needs to be any special association. "Predisposed to schizophrenia" necessarily implies "not neurotypical", and the outcome distribution for individuals who are not neurotypical is much, much broader than neurotypical.

          The pinnacle of success in society has a pronounced overrepresention of neurodivergence, in the same way that pro athletes as a group have freak physical genetics.

          But I would expect that there are equally many people predisposed to schizophrenia who, rather than overachieving prior to symptom onset, end up dysfunctional and battling a variety of substance addictions.

          (and also I'd expect that the relative probability of these outcomes is highly affected by the strength of support networks and socioeconomic status)

        • By ReactiveJelly 2023-11-1519:581 reply

          I had an episode of delusional schizophrenia in my early 20s and luckily haven't relapsed. No hallucinations, just started to think everything was secretly talking about me or to me.

          My pet theory is something like, my brain's dials for "avoid risk" and "recognize patterns" are turned up too high. So I breezed through a software engineering degree without ever partying, but I spend a lot of my time sitting in my house unable to motivate myself to go outside, and I'm not very empathetic (other people's words) and not very outgoing.

          It's not that schizophrenia makes you smart, but that "smart" and "schizophrenic" are both functions of some high-dimensional space, and the same underlying differences can easily cause both.

          On the other side, I have an elder relative who has paranoid schizophrenia and below normal IQ. Us in the tech industry are definitely going to get survivor bias from the "Beautiful Mind" cases around us.

          And of course sometimes you meet those people who are smart, beautiful, rich, and friendly, with no downsides, and all you can think is ... "You son of a bitch" :P

          • By 082349872349872 2023-11-1521:08

            "Recognize patterns" on high is usually an asset in our line of work.

            After reading https://en.wikipedia.org/wiki/Logicomix I wondered how many of {Cantor, Frege, Gödel, Hilbert, Moore, Poincaré, Russel, Turing, Whitehead, Wittgenstein} would —given a modern DX— have been said to be "on the spectrum".

      • By chiefalchemist 2023-11-1521:25

        Some were wrong, but plenty were simply ahead of their time, at least from the perspective of the internet "fad" becoming a ubiquitous mainstream phenomenon.

        Sure, looking back some of the ideas look silly. But when you look at where were are today and the wide range of what's popular and sustainable, some of that looks silly as well.

    • By sodapopcan 2023-11-1516:22

      I just deployed to my neocities site then came and saw this comment :D

      I almost got off of NeoCities recently because I thought I wanted to start adding dynamic parts to my website, but as history has shown me, whenever I start doing that I fall down a rabbit hole and get nothing done. So I buckled down and figured out how to overcome some stuff that was driving me nuts about Hugo and I'm back at it!

      NeoCities definitely has a yonger-feeling crowd for the most part, but I quite like it. It's nice having the feed and discovering all the weird stuff people put on their sites. It does very well at bringing back the feeling of GeoCities. I also love how someone brought back the 88x31 buttons!

      I also really appreciate the Sinatra + Sequel backend :)

    • By StableAlkyne 2023-11-1516:10

      I just love how Neocities has webrings. They were such a great way to find content related to the site you're currently viewing

    • By marginalia_nu 2023-11-1516:01

      I really like the stuff happening over at neocities :-)

      Out of curiosity, do you make any metadata available? Would be a very interesting resource to have, working on making the rest of the web discoverable as well ...

    • By rchaud 2023-11-1521:50

      > but if you build a good site with good content, people do just magically show up through mechanisms

      This hasn't been true for a long time, thanks to social media downranking posts with external links, and Google downranking any site that doesn't post daily updates or heaven forbid, doesnt have SSL enabled.

      A good site with good content takes time and effort to produce. And even then it will simply act as a feeder for people who will regurgitate the same information in simplified terms on a content blog (without a backlink of course) or social media. Worst case scenario, they'll try to productize that knowledge that was made available for free.

      After this happens enough times, people simply stop maintaining those sites.

    • By jxramos 2023-11-1516:411 reply

      I'd love to see a curated awesome-list tagged github project "Awesome Small Web" to peruse.

      https://github.com/topics/awesome-list

      • By arromatic 2023-11-1518:06

        A awesome webring will be a great addition too

    • By sodapopcan 2023-11-1516:55

      PS: I think shouting out neocities on HN just brought it down, lol. IT HAPPENS.

    • By cabalamat 2023-11-1613:44

      Is it possible to upload pages to neocities programmatically? I know you have a Ruby-based program to do so, but can i do it by ftp, http, or something similar?

      The reason i ask is I've written (in Python) wiki software catwiki[1] that allows you to write wiki pages in Markdown. At some point I'd like to extend the program to generate a static site based on the contents of the wiki, and it would be nice to be able to automatically upload it to neocities.

      [1]: https://github.com/cabalamat/catwiki_p3

    • By oalae5niMiel7qu 2023-11-1519:06

      > To paraphrase @izs "if you build it, they will come", is a misquote from a Kevin Costner movie about baseball ghosts

      That movie was a remake of a much older movie.

    • By bluGill 2023-11-1519:071 reply

      What I put on facebook is the type of thing I don't want to get wide reach. I don't want just anyone to see pictures of my kids - that is semi private information that I only want my friends and family to see (and you don't want to see them anyway because you don't know me)

      • By mbrameld 2023-11-1519:251 reply

        > What I put on facebook is the type of thing I don't want to get wide reach. I don't want just anyone to see pictures of my kids - that is semi private information that I only want my friends and family to see (and you don't want to see them anyway because you don't know me)

        That's a different use case than what the GP is describing. Many people use social media, including Facebook, as a platform to build an audience of strangers.

        • By bluGill 2023-11-1520:51

          I know, but Facebook is terrible for that purpose and so I block anyone trying.

    • By cyrialize 2023-11-1716:33

      Big fan of neocities, my personal website is on there! The neocities gem makes it very easy to update the site.

      It's just two commands for me:

      - jekyll build to build _site

      - neocities push _site to recursively upload modified files in _site

    • By vouaobrasil 2023-11-1517:15

      Thanks for the link. I've been looking for something like that!

  • By ploum 2023-11-1515:076 reply

    Not so long ago, every blog has a "blog roll", a list of author’s favorite blogs.

    Nothing was better than have your blog in the blogroll of a "famous" blogger.

    It is funny how people who didn’t live through this blog era are now reinventing it spontaneously. It’s a bit like bloggers were onto something 20 years ago, before being killed by the advertisements monopolies.

    But there’s a big difference between old blogosphere and current blogosphere : old blogs had ads. Most bloggers were experimenting with it, one way or another. We were lured by monetization and killed ourselves in the process.

    Younger bloggers seem to have learn about it: let’s do the same old blogs but, this time, without any ads and by actively preventing tracking.

    That’s how evolution works, when you think about it. It’s beautiful.

    • By prmoustache 2023-11-1515:332 reply

      Before the blogs, many websites had a links section as well.

      + the webrings.

      • By StableAlkyne 2023-11-1520:18

        Links sections were awesome, and made the web feel deeper than it really was. You could go on dives just clicking through and finding so much cool stuff. Plus if it was a hobby site, there was inherently some level of curation - I don't think anybody would be linking to any of the hundreds of lookalike SEO "blogs" nowadays if it weren't for search engines allowing themselves to be gamed.

        Nowadays if it's not on the first "page" of Google (well, whatever the first group of infinite scrolling results is called) it might as well not exist. Makes the web feel flatter and less like a, well, web.

      • By 101008 2023-11-1518:27

        Webrings and some called them "Affiliates" (I dont know where the name came from, it makes more sense in Spanish, not sure in English), but they had this 82x32 buttons on the sidebar (sometimes anitmated GIFs) to similar websites, usually websites handled by friends.

        Oh, internet was so muuch better.

    • By Finnucane 2023-11-1515:27

      Blogrolls could be either very freeform, or very topical, depending on the blog and the blog author, but they did the job--if you liked that blog, chances were pretty good you'd find something interesting in the blogroll.

    • By piperswe 2023-11-1517:58

      I do still see "ads" on these sorts of blogs, but not at all the same type of ad as elsewhere. There are a few "ad networks" that are just free promotion for various other web revival sites, e.g. https://wsmz.gay/#misc-bannerlink

      I quite love it, especially when it fits with the site's aesthetic.

    • By never_inline 2023-11-1516:53

      I planned to do something similar to it on my blog. The idea being using that page as a public bookmark list. It would contain anything from books to blog posts to youtube videos.

      https://mahesh-hegde.github.io/readings/

      But I am not a prolific blogger and haven't updated it anyway for a long time.

    • By LAC-Tech 2023-11-1523:24

      Not so long ago, every blog has a "blog roll", a list of author’s favorite blogs.

      I remember this but I also remember it wasn't called this. (Because in my local English that sounds like bog roll, meaning toilet paper, so I'm sure that would have stuck).

    • By fomine3 2023-11-164:011 reply

      Ads on blog for bonus profit is okay. Blog for ads profit broke the internet.

  • By vasco 2023-11-1515:034 reply

    I tend to think these articles which have become common come from a good place but say more about someone's internet habits than about the internet. I find most social media have a profile section for "personal website". I find many such personal websites by following people's github profiles from interesting repositories or PRs. Sometimes they link to other websites. I do the same in HN, snoop around to see if an interesting comment has a link to a website in the profile. Many articles are posted on HN from personal websites, which again usually link to other websites. I don't know I feel like, if I wanted I could spend all day doing this and would have no problem finding more than hours in the day. So are we complaining about the internet or that we got stuck in the walled gardens of youtube and tik tok and so on and kind of wish we would spend more time on the "old school internet" but don't because the other part is so addictive?

    • By Liquix 2023-11-1515:201 reply

      Just like you said - being surrounded by interesting people on interesting platforms who are likely to create small websites, we occasionally stumble across a link in a walled garden profile.

      For the average internet user, small websites don't exist. Very few Instagram and TikTok profiles feature links to handcrafted sites. Google increasingly funnels all queries to the same 500 giant SEO'd sites.

      • By janalsncm 2023-11-1519:211 reply

        I guess the question is whether that matters. The average internet user today looks a lot different from the average internet user 20 or 30 years ago. The internet looked different but the demographics looked different as well. The average internet user primarily uses Instagram, Facebook, YouTube and TikTok because that is the internet they enjoy.

        • By Gud 2023-11-166:351 reply

          At the same time, the behemoths who operate these platforms are the ones who ruined the open web with their tracking and advertisements.

          • By janalsncm 2023-11-168:59

            I have a tiny website. It’s a static site where I occasionally post things. I don’t have any trackers or ads.

            Behemoths didn’t kill the open web. They created walled gardens which were more attractive to the 99% of people who aren’t interested in doing their own dev ops. Most people prefer to spend their time with their hobby rather than figure out how to set up a server or create ssh keys for GitHub pages.

    • By piva00 2023-11-1515:152 reply

      In my point of view what's lacking is more places where curators that have found interesting small sites can showcase them.

      I used Digg a lot for that, StumbleUpon was also really nice for this type of discovery, then early Reddit had a similar effect.

      Nowadays? I don't know where to go, I can do all this effort of clicking around to find them but honestly I don't have the time, I'm in my mid-30s, I won't be jumping around hyperlinks searching for breadcrumbs of potential good content... A lot of people are doing that work already, like you, we are lacking a good place where we can pool this curation work collectively so others can discover it.

    • By Karrot_Kream 2023-11-1518:06

      I have more Matrix and Discord rooms/"servers" than I can read in a day. If I catch up to the chatter in my Matrix anime rooms and my Discord RPG servers I'm not going to get any work done, any work done on the RPG I'm running, or any chores done at home. This is nothing to say about the personal blogs I read, the substacks, and Reddit, HN, etc. People are starving for personal content? As you say, I think this is more a user problem.

    • By confd 2023-11-1517:40

      "Small web" advocates may eschew the ethics of larger platforms, but they appear to desire the same positive feedback loops that make the larger platforms addicting.

      I think that a problem for some people is that the straightforward solutions to discoverability, such as simply browsing the web in the manner you described or even what is given in the marginalia.nu article, do not solve the desire to be seen as urgently as more technologically coordinated processes.

HackerNews