A privacy-first, self-hosted, fully open source personal knowledge management software, written in typescript and golang. - siyuan-note/siyuan
SiYuan is a privacy-first personal knowledge management system, support fine-grained block-level reference and Markdown WYSIWYG.
Welcome to SiYuan English Discussion Forum to learn more.
Most features are free, even for commercial use.
siyuan://Some features are only available to paid members, for more details please refer to Pricing.
It is recommended to give priority to installing through the application market on the desktop and mobile, so that you can upgrade the version with one click in the future.
Mobile:
Desktop:
The easiest way to serve SiYuan on a server is to deploy it through Docker.
b3log/siyuanThe overall program is located under /opt/siyuan/, which is basically the structure under the resources folder of the Electron installation package:
The entry point is set when building the Docker image: ENTRYPOINT ["/opt/siyuan/entrypoint.sh"]. This script allows changing the PUID and PGID of the user that will run inside the container. This is especially relevant to solve permission issues when mounting directories from the host. The PUID (User ID) and PGID (Group ID) can be passed as environment variables, making it easier to ensure correct permissions when accessing host-mounted directories.
Use the following parameters when running the container with docker run b3log/siyuan:
--workspace: Specifies the workspace folder path, mounted to the container via -v on the host--accessAuthCode: Specifies the access authorization codeMore parameters can be found using --help. Here’s an example of a startup command with the new environment variables:
docker run -d \
-v workspace_dir_host:workspace_dir_container \
-p 6806:6806 \
-e PUID=1001 -e PGID=1002 \
b3log/siyuan \
--workspace=workspace_dir_container \
--accessAuthCode=xxxPUID: Custom user ID (optional, defaults to 1000 if not provided)PGID: Custom group ID (optional, defaults to 1000 if not provided)workspace_dir_host: The workspace folder path on the hostworkspace_dir_container: The path of the workspace folder in the container, as specified in --workspaceaccessAuthCode: Access authorization code (please be sure to modify, otherwise anyone can access your data)To simplify things, it is recommended to configure the workspace folder path to be consistent on the host and container, such as having both workspace_dir_host and workspace_dir_container configured as /siyuan/workspace. The corresponding startup command would be:
docker run -d \
-v /siyuan/workspace:/siyuan/workspace \
-p 6806:6806 \
-e PUID=1001 -e PGID=1002 \
b3log/siyuan \
--workspace=/siyuan/workspace/ \
--accessAuthCode=xxxFor users running Siyuan with Docker Compose, the environment variables PUID and PGID can be passed to customize the user and group IDs. Here's an example of a Docker Compose configuration:
version: "3.9"
services: main: image: b3log/siyuan command: ['--workspace=/siyuan/workspace/', '--accessAuthCode=${AuthCode}'] ports: - 6806:6806 volumes: - /siyuan/workspace:/siyuan/workspace restart: unless-stopped environment: # A list of time zone identifiers can be found at https://en.wikipedia.org/wiki/List_of_tz_database_time_zones - TZ=${YOUR_TIME_ZONE} - PUID=${YOUR_USER_PUID} # Customize user ID - PGID=${YOUR_USER_PGID} # Customize group IDIn this setup:
PUID and PGID are set dynamically and passed to the container1000 will be usedBy specifying PUID and PGID in the environment, you avoid the need to explicitly set the user directive (user: '1000:1000') in the compose file. The container will dynamically adjust the user and group based on these environment variables at startup.
In the image, the entrypoint.sh script ensures the creation of the siyuan user and group with the specified PUID and PGID. Therefore, when the host creates a workspace folder, pay attention to setting the user and group ownership of the folder to match the PUID and PGID you plan to use. For example:
chown -R 1001:1002 /siyuan/workspaceIf you use custom PUID and PGID values, the entrypoint script will ensure that the correct user and group are created inside the container, and ownership of mounted volumes will be adjusted accordingly. There’s no need to manually pass -u in docker run or docker-compose as the environment variables will handle the customization.
Use NGINX reverse proxy to hide port 6806, please note:
/wsPUID and PGID environment variables match the ownership of the mounted directories on your host systemNote: First run chown -R 1000:1000 /mnt/user/appdata/siyuan in the terminal
Template reference:
Web UI: 6806
Container Port: 6806
Container Path: /home/siyuan
Host path: /mnt/user/appdata/siyuan
PUID: 1000
PGID: 1000
Publish parameters: --accessAuthCode=******(Access authorization code)
We release insider preview before major updates, please visit https://github.com/siyuan-note/insider.
See Development Guide.
The data is saved in the workspace folder, in the workspace data folder:
assets is used to save all inserted assetsemojis is used to save emoji imagessnippets is used to save code snippetsstorage is used to save query conditions, layouts and flashcards, etc.templates is used to save template snippetswidgets is used to save widgetsplugins is used to save pluginspublic is used to save public data.sy in the notebook folder are used to save the document data, and the data format is JSONData synchronization through third-party synchronization disks is not supported, otherwise data may be corrupted.
Although it does not support third-party sync disks, it supports connect with third-party cloud storage (Member's privileges).
In addition, you can also consider manually exporting and importing data to achieve data synchronization:
SiYuan is completely open source, and contributions are welcome:
For more details, please refer to Development Guide.
You can Check update in Settings - About - Current Version, or pay attention to Official Download or GitHub Releases to get the new version.
The first sub-block under the list item is the block icon omitted. You can move the cursor into this block and trigger its block menu with Ctrl+/ .
If the data repo key is correctly initialized on multiple devices before, the key is the same on all devices and can be set in Settings - About - Data repo key - Copy key string retrieve
If it has not been configured correctly before (for example, the keys on multiple devices are inconsistent) or all devices are unavailable and the key string cannot be obtained, you can reset the key by following the steps below:
Most features are free, even for commercial use.
Member's privileges can only be used after payment, please refer to Pricing.
The birth of SiYuan is inseparable from many open source projects and contributors, please refer to the project source code kernel/go.mod, app/package.json and project homepage.
The growth of SiYuan is inseparable from user feedback and promotion, thank you for everyone's help to SiYuan ❤️
Welcome to join us and contribute code to SiYuan together.
Looks cool, but as I have been on this knowledge management / productivity journey like everybody. Here are my findings:
If you are reasonably comfortable with computers / Unix.
- You need to first rely on a directory structures, filenames, plaintext, lists and maybe markdown. Stick with a "File over app", Unix approach.
- Try to sort things with universal concepts: locations, things, people, events, metrics, howtos. A bit like the 5Ws approach.
- Leverage good Unix tools: unix commands, make/justfiles, (rip)grep, git, fzf, etc.
- Do not try to solve the problem through the Web. Because you will end up trying to solve web problems instead of basic knowledge management and productivity issues.
- The smartphone/touchscreen is a major problem, but as with the Web do not try to solve it. Use your file manager or even fzf in termux can be adapted to be reasonably usable on a touchscreen.
Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy.
I've been experimenting with Flatnotes (https://github.com/Dullage/flatnotes) for a while and really like the design. No notebooks or even folders, just a single directory with markdown files and decent search & tagging. It feels a lot like what happened to email when we gave up all the up-front structuring with deep hierarchies and just said index it and we'll find it when we need to.
The project is just "good enough" for what I need, and aside from tiny bugs whenever I find a gap I can either work around it or live without. Constraints are a powerful motivator for both creativity and getting stuff done.
Tagging is a strictly more powerful tool than hierarchies, at least with same amount tags vs directories/categories, because an item can be tagged using multiple different tags, but can only be in one directory, unless you create duplicates or symlinks or whatever.
I find the opposite. Being forced to consider hierarchical categorisation leads to a more powerful system in practice. It helps me create a mental reference to the item being stored. And it often causes me to see a better way to form an item in the first place — eg maybe this note is really two notes, maybe this idea can actually just be discarded, etc. Or, on rare occasions, a new item doesn’t fit anywhere (but is important) so I need to tweak the hierarchy itself — and that’s a good thing, once in a while, as it can lead to creative insights about the domain as a whole.
I’ve found tagging systems usually become a kind of dark swamp where things go never to be seen again. The lack of structure means I have little memory of what’s gone into the system, so I end up with too much duplication of ideas and inconsistency of style (mess). All this makes it uninviting and difficult to ‘explore’, so I don’t use it much except as a dumping ground, and the swampiness compounds over time.
Hierarchy doesn’t make sense if you care about multiple dimensions, which I often do with notes.
Sometimes I want to look at all notes relating to c++, sometimes everything related to a personal project. Directories don’t support that without symlinking everything that mentions c++ into a folder for that.
Hierarchy can do that too, once you allow a single file to exist in multiple hierarchies at once. Think of database, we have a single table, yet we have a lot of different indexes to search on.
We could do the same, either with symlinks or hardlinks
Everything is a hierarchy, even the tags in your mind do belong under some tree/node like imagination. Tag is essentially just what you described "duplicates or symlinks or whatever."
This isn't really true. Even mature, sophisticated attempts at making hierarchical classification systems (e.g. the UDC[0]) don't claim to have created a system where each document gets a unique leaf on a tree, and indeed instead attempt provide some syntax for adding additional metadata into its "nodes," i.e. the nodes become the sum of their tags.
[0] https://en.wikipedia.org/wiki/Universal_Decimal_Classificati...
> but can only be in one directory, unless you create duplicates or symlinks or whatever.
Non-sym links do exactly this. Starting with your file in foo;
ln foo/file bar/file
will have the file (really a reference-counted pointer) appear in both directories. It's the same location on disk and there's no overhead until you run out of inodes.I really like tagging but somehow the concept could never become central to any filesystem:
- I believe BeOS tried,
- MS tried with WinFS but cancelled it in Windows Vista,
- I am sure some cloud storage service bet on it but can't cite any.
Actually tagging is probably mainly successful in cases where we can reliably automate the tagging such as email or photos.
MacOS has had first-class support for file tagging for years: You can add arbitrary tags either from “Save as” dialogs or from Finder, and can then browse by tags in Finder or search through your file system filtered by tags.
Does the native file tagging see much adoption? I'm a lifelong Mac user and tried using the tags when they first launched, but eventually it was too much work and I resorted to using search to find what I'm looking for. Not macOS native search but tools like LaunchBar, Alfred, et al or apps like EasyFind.
Colours, yes – but then again I worked in a print shop that'd been using Apple since MacOS 8, so they might be used to those quirks.
I like tagging for bookmarks in Firefox and derivatives of Firefox.
Sharepoint tries to push people towards tags, but doesn't always succeed.
Obsidian lets you live on top of the filesystem structure using markdown files, and it gives you links, tags, topic clouds, and more.
If you use it with reasonable conventions, you can compile and deploy a web version with ghost or zola or something automatically run in CI.
It's backed up, synced, and comes with desktop and mobile apps. It has vim key bindings and git plugins.
It's a pretty sweet system that works nicely with Unix, and it adapts to multiple navigational and organizational paradigms.
I've tried a lot of things, and nothing has ever come close to Obsidian.
I have gone through the whole journey similar to you as well. It is so frustrating to keep searching for that perfect app only to find that every other app lacks something or the other. For the last year or so, I've settled on just plain markdown and the directory structure for hierarchy just like you.
I use my preferred editor (Emacs) to modify the files. I get to use all the functionality like key-bindings, search, version control. I push my notes to my self-hosted Forgejo instance (but it can be Github). Forgejo already has a web-interface the notes and links just work there.
On my laptop, I have mdbook configured to watch the directory and build the notes into static website. So if I just want to go through by notes or read something I can use that as well.
I tried finding solutions to take notes on my phone. But I realised that if its longer than a few lines than I prefer using my laptop anyways. I only ever read my notes on the phone. I've setup the same mdbook behind VPN so I can access it on my phone as well. If I ever want to modify notes on-the-go then I can use the Forgejo web interface as well.
You should check out the HyWiki part of the Emacs Hyperbole hypertext package available from melpa or elpa-devel. It utilizes Org mode for notes and WikiWords to automatically interlink them as you type. The WikiWords work in any kind of text buffer as well, even in programming file comments. It even lets you link to many other kinds of Emacs entities like bookmarks, Org IDs and headings, etc. One command to build a web-based wiki. With a little practice, it might amaze you.
This is exactly the conclusion I came to. I still use Obsidian for a lot of things, but I've developed my own task management software that uses plain text files and fzf for everything: https://codeberg.org/ngp/tsk
It has worked great for me! That being said, sync and mobile usage is still a bit of a sore spot with it. It works beautifully with Termux, but I wish there was some way to slap a basic UI on top of a CLI application for mobile. Tk/Tcl is the closest but there's only options on Android (I'm mostly an iOS user) and even then it's not really ideal. For sync, I at least have a reasonable plan: IMAP4 or git-based sync. The roadmap and tasks for the project are tracked in-repo with itself, so it definitely works.
> but I wish there was some way to slap a basic UI on top of a CLI application for mobile
I have only relied on fzf so far but have played with the various components of the Termux:API [0] for basic UIs, and it works well.
There is also Termux:GUI [1] with Bash bindings which allows to build more advanced UIs.
I'll have to look at Termux GUI again. I've tried poking around with Tasker and it's Termux plugin but it seems so confusing and poorly documented that I'd rather just write a native app at that point lol
If you want some windows-forms-esq app development, I highly recommend checking out Flutter Flow. It isn’t exactly cheap, but I’ve used it for some basic personal apps.
I kinda doubt FlutterFlow would meet my needs, it looks like it's primarily build on web technologies which is a nonstarter for this.
As far as I know, it's the opposite (using Skia to render). Flutter for the web is a second-class citizen, whereas for mobile apps, it's quite productive.
FlutterFlow is just the UI to build Flutter apps, which yes, is built on web-tech. However, Flutter runs natively on mobile devices.
I'll leave my imcomplete mind dump below for posterity but I'll make it known that while writing it, I did realise that what I've invented is a text editor.
----------------------------------
I had an idea for a deeply unix-y note taking software a while ago and I've continued to think it would actually work very well.
Notes could either be an sqlite database (nets you compression, performance, easy backups, etc.) or just text files on the filesystem (easy searching, less lock-in, whatever). The "editor" just creates a new temporary file and opens it in `$EDITOR`. You can optionally name it after, otherwise it's named the current date.
Want linking? Use a filesystem-aware editor. Want syntax highlighting? Use an editor with syntax highlighting. Want to sync your notes? Combine it with Syncthing. Want to search your notes? fzf.
I do this except with git, fzf, and a simple CLI tool:
Contributions welcome :)
Or, you could just fire up Zim Desktop Wiki [1], a local application you can run on any one of the major OS, and be mostly done with it. It gives you bullet 1, 2, 3[2], 4, and backlinks out of the box.
You get 5 if you are willing to compromise. Since it's just all text files in directory trees, you can sync the tree to your mobile phone (e.g with syncthing) and just use any text editor.
[1] https://zim-wiki.org/index.html [2] git with an add-on.
After using Bear.app and Things.app for a while, I've moved firmly in the file over app camp. Apps can be enjoyable to use until you don't have access to the app anymore (platform, longevity,...). Now I'm using Emacs with org files, but could just as well use any tools that favors files. Any novel use case I want, I can script it out in a moment, while still enjoying the benefits of viewing my information on most platforms.
> Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy
The only thing you need is some kind of structure, like the wiki link format or org link format, and you could 'rgrep' the notes directory for a particular link (vim and emacs can put the result in a buffer).
I just use TexStudio. I don't use proper tex files, but it never complains unless I were to try and compile. But their navigation tree and tabbed interface for multiple files makes organizing things really nice.
I can have multiple instances at once, and it's all managed via files and my filesystem.
Eventually I want to build a web frontend to index and reference my data, but for now the above approach is working well.
Unlike Obsidian, TexStudio is open-source, which I consider a plus.
Great suggestions.
I'm quite happy with Obsidian on Linux, especially due to great plugin ecosystem.
Granted that it would have been great if the clients were open-source; but Obsidian is one of those non-FOSS apps which are so consumer oriented that they get an exception from me.
I sync the files from Obsidian using syncthing to Android & use Markor for editing. But this wouldn't be possible on iOS.
For backlinks, have a look at Andy Matuschak's note-link-janitor: https://github.com/andymatuschak/note-link-janitor/
I wrote my own version of it a few years ago, but have moved on, and currently use LogSeq instead: https://github.com/cdaven/noteexplorer
For a long while I was trying to keep text notes in a structured set of directories, Tombo.exe made that a lot easier/nicer to use:
sadly, it looks to have dropped off the 'net.
All falls Short when it comes to cross referencing, sub notes , rich capabilities.
What do you mean by "the Web"? The internet, or some sort of network graph of notes (such as can be made in Obsidian)?
Either way, what is the problem with it?
org-roam gives you backlinks. directory of text files.
>Something I have been wondering about is the "backlink" feature.
I trust this is why everyone tries to rely on network-like structure.
I've been using this for a couple of years on my home-server.
It's an Obsidian knock-off. It's pretty janky and the documentation is lacking. It's open-source which is nice... But the company behind it is ??? I don't know. They are Chinese but I couldn't find much about them.
I use it because I can self-host it, it has most of the features Obsidian has, and I can use it in a web-browser from anywhere - which is the biggest feature for me that Obsidian lacks.
It seems like they borrowed heavily from Notion, Obsidian and RemNote, as far as I can tell (wouldn’t call it a knock-off though, since there are sooo many apps in this space that you don’t really know who came up with what anymore). But the app doesn’t feel janky to me at first glance, it definitely feels more responsive than Notion and less “slippy” than RemNote. Although it is quite noisy with all the tooltips popping up immediately.
My first impression is that they really wanted to include everything (even RemNote-like spaced-repetition flashcards, Notion-like Databases and of course there has to be AI too) and it seems like they did a pretty decent job at that. I also appreciate that there are so many export options, even for Org-Mode (preserving internal links, images, code-blocks, etc.).
I like that it provides a solid, feature-rich alternative to all the cloud-first, closed-source apps in this space. But it may be too distracting/overwhelming for my use-cases with all the advanced layout capabilities and features though. Tana is a similar all-in-one solution that is really well done (and more innovative than this one), but I always seem to gravitate toward more focussed apps.
I assume you want to self host in order to sync locally, right? If that's the case, you can use Obsidian with git plugin and you get sync + versioning (all what git offers).
Which git plugin do you like? And have you used the plugin from more than one machine to a repo more than one machine also syncs with?
Sync is premium since some versions. Before it was free, you had only to provide a S3 bucket or a webdav location
I use syncthing to sync my obsidian vault which is also a git repository between linux, mac and windows.
Works great!
Logseq is floss(gnu license, closurescript) and almost perfect for me but it's totally mysterious to me what's stopping it from serving a web interface that serves files on the web server. It's an electron app with an HTTP API but for some reason the web demo only opens files clientside with filesystem API
Anyway, one of these days I'll fork it and make it work for me. I also have a perverted desire to change the serialization format from markdown to XML so I can manipulate the graph with other tools that talk xpath, xquery, basex.
They are doing a rewrite of the app to have a SQLite DB storage for notes instead of markdown files. I think once that is released it will be available in a web browser. Currently in alpha testing, I access the application through the browser.
I also installed it on my computer to give it a try, but then as you mentioned I could not find who are behind it other than it's based in China. So decided to just keep with Obsidian
Took me like a minute to find out who are behind it. Committers on GitHub project point to two profiles:
- https://github.com/Vanessa219
The first profile has a README (zh-CN, easily translated) introducing themselves as a married couple as well as their career trajectory leading to their current company. Googling the company name leads me to https://www.tianyancha.com/company/3153162387 showing the company’s legal structure, the legal names of the couple, their address, etc. (again, zh-CN but easily translated). Looks like I can view their financial reports too if I have a subscription.
The profiles also link to their social media accounts (on their own dev-focused community).
What more is there to know? At least it’s more than the average ShowHN asking for your email and sometimes credit card. I don’t understand these “couldn’t find much about them” claims.
The link you posted gives me "According to relevant legal regulations, access is temporarily not supported in your current location." and "If your device or the Wi-Fi environment you are in is using a VPN service, please disable it and try again."
I'm not using any VPN. Normal internet from Germany
It is also unavailable from Canada.
I vaguely recall this being part of a tit-for-tat thing between China and the anti-Chinese. There have been movements to restrict Chinese access to FOSS, because forking FOSS lowers Chinese dependence on the West, along with (ironic) accusations that the "authoritarian" Chinese are limiting access to Western tech products. I thought there was some sort of legislative or judicial outcome that came out of it, but no luck with a quick google.
-----
U.S. restriction on Chinese use of open-source microchip tech would be hard to enforce - October 13, 2023
> U.S. lawmakers are pressuring the administration of President Joseph Biden to place restrictions on RISC-V to prevent China from benefiting from the technology as it attempts to develop its semiconductor industry.
https://thechinaproject.com/2023/10/13/u-s-bar-on-chinese-us...
-----
China’s Use of Foreign Open-Source Software, and How to Counter It - April 2, 2024
> Democratic governments also need to reassess which products should not be made open-source because they’re at risk of being weaponized by malign actors.
https://chinaobservers.eu/chinas-use-of-foreign-open-source-...
-----
Whatever the US did, Europe would do. Anybody in the US or Europe working on a FOSS project with Chinese contributors that they're friendly with? Has anything happened recently?
TianYancha is a corporate data aggregation website, it has nothing to do with FOSS. Your post is such a clumsy attempt to steer the conversation into Anti-Americanism/Westernism. Like really blatant lol.
And Australia.
Okay yeah appears to be heavily geolocked. Still a lot of information otherwise.
They open sourced it and you can self hosted. I mean, does it even matter where they are from? Why it’s automatically suspicious when you know the authors are Chinese.
True the origin does not matter, but it would be better to have more transparency about the contributors even if it's an open source tool. Because you can still get injected with a malicious code when an update is pushed.
But I agree we should not categorize according to geolocation, but more transparency would be better irrespective of the location in any project.
Why are you not using Obsidian itself? Anything wrong with it?
for web-browser access I host Obsidian with kasmvnc. It is not ideal for use on the phone, but from a tablet it works, and I can host it with Vivaldi in a webpanel for quick edits.
I keep seeing these note-taking tools pop up again and again. I am heavily invested into Logseq already and the new database version is coming out soon. Unless these tools provide import and export utilities to convert your notes between the most popular applications like Logseq and Obsidian, only new users will use them and people with very few notes who can put in the time to move.
Creating a note taking, personal productivity app is an elaborate but common procrastination technique for developers.
After forcing myself to learn and adapt to some note-taking system, I too didn't find it useful for me yet. But i keep pushing myself through because I still believe that there must be some value that I could take from taking notes. Just I didn't find a system that suits me well...? One thing that i find the real benefit/value from all this learn and adapt is "writing as a way to think" (is it by feynmann?). When doing complicated work, writing really help to ease your cognitive load and help you find the gaps in your line of thought. But, I couldn't find a suitable method when dealing with general/every day note-taking. I still have that "graveyard for thought" problem when writing general notes
right up there with creating your own personal website
At least you didn't create a new language for your Personal Home Page and even later rewrote it to be some sort of general hypertext processor.
I feel exposed lol
What is the benefit of the database version? I like that the backend is in text files. I guess there could be some performance/compression benefits to be had, but given the amount of writing a mortal is likely to do over a lifetime, I am not immediately sold on the idea.
I believe the main goal is improved speed for huge vaults.