diff --git a/content/posts/2023-11-11-index.md b/content/posts/2023-11-11-index.md index 986e071..3e9c26e 100644 --- a/content/posts/2023-11-11-index.md +++ b/content/posts/2023-11-11-index.md @@ -1,9 +1,9 @@ +++ title = "Declarative filesystem management with Emacs & Org Mode" author = ["Pavel Korytov"] -date = 2023-11-10 +date = 2023-11-11 tags = ["emacs", "orgmode"] -draft = true +draft = false +++
@@ -21,39 +21,39 @@ My filesystem is, shall we say, not the most orderly place. -It's been kinda messy, and messy in different ways across my three machines. For instance, my laptop had work projects in `~/Code/Job`, my work machine had just `~/Code`, and so on. +It's been somewhat messy, and messy in different ways across my three machines. For instance, my laptop had work projects in `~/Code/Job`, my work machine had just `~/Code`, and so forth. -And it's strange that I wasn't able to find any existing solution to that problem. I can't be the only one with that problem, can I? +Strangely, I couldn't find and existing solution to that problem. Surely, I can't be the only one facing that issue, can I? -Anyway, I'm lucky to know my way in (make-yourself-a) Swiss Army Knife of computing called [Emacs](https://www.gnu.org/software/emacs/), so... below is my attempt to make something of it. And another entry to add to the already substantial list of my Emacs uses. +Fortunately, I'm well-acquainted (make-yourself-a) Swiss Army Knife of computing called [Emacs](https://www.gnu.org/software/emacs/), so... below is my attempt to make something of it. And another addition to the already substantial list of my Emacs uses. -Also, my `M-x magit-log-buffer-file` shows I've created that file on the same day I had written the embedded toot, so this must be the longest Emacs thing I've been figuring out. And it's also probably the least portable, but I nevertheless hope you'll find it useful. +Also, my `M-x magit-log-buffer-file` shows I've created that file on the same day I had written the embedded toot, so this must be the longest Emacs thing I've been figuring out. And it's probably the least portable, but I nevertheless hope you find it useful. ## Idea {#idea} {{< figure src="/images/index/index.png" >}} -So, I've decided to try declarative filesystem management. +So, I decided to try declarative filesystem management. -At the core, there's my work-in-progress adaptation of [Johnny.Decimal](https://johnnydecimal.com/)[^fn:1]. Essentially, it proposes to prefix your folders with numbers like `12.34`, where: +At the core is my work-in-progress adaptation of [Johnny.Decimal](https://johnnydecimal.com/)[^fn:1]. Essentially, it suggests prefixing your folders with numbers like `12.34`, where: -- the first digit is "[category](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)" -- the second digit is "[area](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)" +- the first digit is the "[category](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)"; +- the second digit is the "[area](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)"; - the last two digits are the [ID](https://johnnydecimal.com/10-19-concepts/11-core/11.03-ids/). -The point is to organize your folder structure and limit its depth, which should make finding things quicker and more straightforward. Check the website for a more thorough description. +The point is to organize your folder structure, limiting its depth for quicker and more straightforward access. Check the website for a more thorough description. -So, what I want is: +So, what I want is to: -- to define a Jonny.Decimal-esque file tree in a single [Org](https://orgmode.org/) file; +- define a Jonny.Decimal-esque file tree in a single [Org](https://orgmode.org/) file; - have different nodes of that file tree active on different machines, e.g. I don't want [my Emacs stuff](https://github.com/SqrtMinusOne?tab=repositories&q=&type=&language=emacs+lisp&sort=) on my work machine; -- use different tools to sync different nodes (as of now [git](https://git-scm.com/), [MEGA](https://mega.nz/), and "nothing"). +- use different tools to sync different nodes (currently [git](https://git-scm.com/), [MEGA](https://mega.nz/), and "nothing"). ### Folder structure {#folder-structure} -As I said, I tried (and still trying) to adapt the proposed scheme to better suit my needs. Here's a subset of my current tree. +As I said, I tried (and still trying) to adapt the proposed scheme to better suit my needs. Here's a subset of my current tree: ```text 10-19 Code @@ -76,11 +76,11 @@ As I said, I tried (and still trying) to adapt the proposed scheme to better sui 33 Library ``` -The root of the tree is my `$HOME`. The entry at the third (or second) level can be either an entity it itself (such as a git repository), or a "project root". +The root of the tree is my `$HOME`. The entry at the third (or second) level can be either an entity itself (such as a git repository), or a "project root". -In several places I use year references (`Y20`) instead of the plain `AC.ID`. This is mainly to group things by academic years, e.g. to find all my publications or students in some year, which I need for occasional reports. I also have semester references (`SEM10`) for my undergraduate studies. +In several places, I use year references (`Y20`) instead of the plain `AC.ID`. This is mainly to group things by academic years, e.g. to find all my publications or students in a specific year, which I need for occasional reports. I also have semester references (`SEM10`) for my undergraduate studies. -Project structure is also more or less standard. Johnny.Decimal [proposes](https://johnnydecimal.com/10-19-concepts/13-multiple-projects/13.01-introduction/) to use `PRO.AC.ID` to manage multiple projects, but this doesn't seem to fit quite as well to my case, so I came up with the following: +The project structure is more or less standard. Johnny.Decimal [proposes](https://johnnydecimal.com/10-19-concepts/13-multiple-projects/13.01-introduction/) using `PRO.AC.ID` to manage multiple projects, but this doesn't seem to fit quite as well in my case. So I came up with the following: ```text 10.03 Digital Trajectories ; project root @@ -101,17 +101,17 @@ Perhaps this is too verbose (`10.03.R.01`), but it works for now. ### Tools choice {#tools-choice} -As I've said, my current options to manage a particular node are: +As I mentioned earlier, my current options to manage a particular node are: - [git](https://git-scm.com/); - [MEGA](https://mega.nz/) - for files that don't fit into git, such as DOCX documents, photos, etc.; -- nothing - something that I don't need to sync across machines, e.g. database dumps. +- "nothing" - for something that I don't need to sync across machines, e.g. database dumps. -One other tool I considered was [restic](https://github.com/restic/restic). It's an interesting backup & sync solution, with built-in encryption, snapshots, etc. +Another tool I considered was [restic](https://github.com/restic/restic). It's an interesting backup & sync solution with built-in encryption, snapshots, etc. -My problem is that its repositories are only accessible via restic. So, even if I use something like MEGA as a backend, I won't be able to use the MEGA file-sharing features, which I occasionally want for document or photo folders. So for now I'm more interested in synchronizing the file tree in MEGA with [MEGAcmd](https://github.com/meganz/MEGAcmd) (and also clean up the mess there, two birds with one stone). +However, a challenge I encountered is that its repositories are only accessible via restic. So, even if I use something like MEGA as a backend, I won't be able to use the MEGA file-sharing features, which I occasionally want for document or photo folders. Hence, for now, I'm more interested in synchronizing the file tree in MEGA with [MEGAcmd](https://github.com/meganz/MEGAcmd) (and also clean up the mess up there). -Another interesting tool is [rclone](https://rclone.org/), which provides a single interface for multiple services like Google Drive, Dropbox, S3, WebDAV. It also supports MEGA, but requires turning off the two-factor authentication, which I don't want. +Another interesting tool is [rclone](https://rclone.org/), which provides a single interface for multiple services like Google Drive, Dropbox, S3, WebDAV. It also supports MEGA, but it requires turning off the two-factor authentication, which I don't want. ## Implementation {#implementation} @@ -132,7 +132,7 @@ And a package called [ini.el](https://github.com/daniel-ness/ini.el) to parse IN :straight (:host github :repo "daniel-ness/ini.el")) ``` -The rest is built-in into Emacs. +The rest is built into Emacs. ### Org tree {#org-tree} @@ -155,8 +155,8 @@ The org tree is located in my `org-mode` folder in a file called `index.org`: Each "area" is an Org header with the `folder` tag; the Org hierarchy forms the file tree. A header can have the following properties: -- `machine` - list of hostnames for which the node is active (or `nil`) -- `kind` - `mega`, `git` or `dummy` +- `machine` - a list of hostnames for which the node is active (or `nil`) +- `kind` - `mega`, `git`, or `dummy` - `remote` - remote URL for `git` - `symlink` - in case the folder has to be symlinked somewhere else[^fn:2] @@ -202,13 +202,14 @@ So, let's parse the Org tree. This is done by recursively traversing the tree re ```emacs-lisp (defun my/index--tree-get-recursive (heading &optional path) - "Recursively read index tree from HEADING. + "Read the index tree recursively from HEADING. HEADING is an org-element of type `headline'. -PATH is the path to the current node. If not provided, it is -assumed to be the root of the index. The return value is an -alist, see `my/index--tree-get' for details." +If PATH is provided, it is the path to the current node. If not +provided, it is assumed to be the root of the index. + +The return value is an alist; see `my/index--tree-get' for details." (when (eq (org-element-type heading) 'headline) (let (val (new-path (concat @@ -241,7 +242,7 @@ alist, see `my/index--tree-get' for details." val))) (defun my/index--tree-get () - "Read index tree from the current org buffer. + "Read the index tree from the current org buffer. The return value is a list of alists, each representing a folder/node. Alists can have the following keys: @@ -268,7 +269,7 @@ folder/node. Alists can have the following keys: #### Verify tree {#verify-tree} -I also want to make sure that I didn't mess up the numbers, i.e. didn't place `10.02` under `11`, and so on. +I also want to make sure that I didn't mess up the numbers, i.e., didn't place `10.02` under `11`, and so on. To do that, we first need to extract the number from the name: @@ -357,7 +358,7 @@ Finally, we need to narrow the tree to only leave nodes that are active for the Next, apply the tree to the filesystem. -I've decided to implement this by generating a bash script and executing it with `bash +x`. This way I can check the required changes in advance and avert potential loss of data if something unexpected happens. +I've decided to implement this by generating a bash script and executing it with `bash +x`. This way, I can check the required changes in advance and avert potential data loss if something unexpected happens. One command for the script will be a list like: @@ -378,14 +379,13 @@ FULL-TREE and TREE are forms as defined by `my/index--tree-get'. TREE is the narrowed FULL-TREE (returned by `my/index--tree-narrow'). ACTIVE-PATHS is a list of paths that are currently active. If not -provided, it is computed from TREE, i.e. as those paths that have to -exists on the current machine. +provided, it is computed from TREE. The return value is a list of alists with the following keys: - path - the path of the folder - exists - whether the folder exists on the filesystem - has-to-exist - whether the folder exists in the tree -- extra - if the folder exists in the filesystem but not in tree. +- extra - if the folder exists in the filesystem but not in the tree. - children - a list of alists with the same keys for the children of the folder." (let ((active-paths (or active-paths (my/index--tree-get-paths tree)))) @@ -587,8 +587,7 @@ To sync git, we just need to clone the required git repos. Removing the repos is "Get commands to clone the yet uncloned git repos in TREE. TREE is a form a defined by `my/index--tree-get'. This is supposed to -be the tree narrowed to the current machine -(`my/index--tree-narrow'). +be the tree narrowed to the current machine (`my/index--tree-narrow'). The return value is a list of commands as defined by `my/index--commands-display'." @@ -866,7 +865,7 @@ for recursive calls. The result is a list of alists with the following keys: - `:names` - list of names, e.g. (\"10.01 Something\" \"10.01.01 Something\") -: `:path` - path to the folder, e.g. +- `:path` - path to the folder, e.g. \"/path/10 stuff/10.01 Something/10.01.01 Something/\" - `:child-navs` - list of child navigation structures (optional)" (seq-sort-by @@ -925,7 +924,7 @@ The return value is a form as defined by `my/index--nav-get'." #### Emacs interface {#emacs-interface} -As for Emacs interface, a plain `completing-read` is sufficient, except that I don't want [prescient.el](https://github.com/radian-software/prescient.el) to interfere with the default ordering of elements. +As for Emacs interface, `completing-read` is sufficient, except that I don't want [prescient.el](https://github.com/radian-software/prescient.el) to interfere with the default ordering of elements. ```emacs-lisp (defun my/index--nav-prompt (nav) @@ -963,7 +962,7 @@ command as follows: - '(4): Select an indexed directory, and select a child indexed directory if available. - If in an indexed directory with indexed children (a project): - - nil: Select another indexed directory from the project + - nil: Select another indexed directory from the project. - '(4): Select a top-level indexed directory (the same as nil for the previous case). - '(16): The same as '(4) for the previous case. diff --git a/org/2023-11-11-index.org b/org/2023-11-11-index.org index 6c92c71..87236f7 100644 --- a/org/2023-11-11-index.org +++ b/org/2023-11-11-index.org @@ -4,7 +4,7 @@ #+DATE: 2023-11-11 #+HUGO_TAGS: emacs #+HUGO_TAGS: orgmode -#+HUGO_DRAFT: true +#+HUGO_DRAFT: false #+begin_abstract The post describes a Johnny.Decimal-inspired filesystem structure, declared in an org file and synchronized across machines. Different folders are available on different machines.