sqrtminusone/sqrtminusone.github.io
1
0
Fork 0
mirror of https://github.com/SqrtMinusOne/sqrtminusone.github.io.git synced 2025-12-10 15:53:03 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions

index: remove draft

Browse source
This commit is contained in:
Pavel Korytov 2023-11-11 13:02:10 +03:00
parent e10cb2ee55
commit 3ee4e2ebf8
2 changed files with 41 additions and 42 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

81
content/posts/2023-11-11-index.md
View file

@ -1,9 +1,9 @@
+++ +++
title = "Declarative filesystem management with Emacs & Org Mode" title = "Declarative filesystem management with Emacs & Org Mode"
author = ["Pavel Korytov"] author = ["Pavel Korytov"]
date = 2023-11-10 date = 2023-11-11
tags = ["emacs", "orgmode"] tags = ["emacs", "orgmode"]
draft = true draft = false
+++ +++
<div class="abstract"> <div class="abstract">
@ -21,39 +21,39 @@ My filesystem is, shall we say, not the most orderly place.
<iframe src="https://emacs.ch/@sqrtminusone/110514686718545191/embed" class="mastodon-embed" style="max-width: 100%; border: 0" width="500" allowfullscreen="allowfullscreen"></iframe><script src="https://emacs.ch/embed.js" async="async"></script> <iframe src="https://emacs.ch/@sqrtminusone/110514686718545191/embed" class="mastodon-embed" style="max-width: 100%; border: 0" width="500" allowfullscreen="allowfullscreen"></iframe><script src="https://emacs.ch/embed.js" async="async"></script>
</center> </center>
It's been kinda messy, and messy in different ways across my three machines. For instance, my laptop had work projects in `~/Code/Job`, my work machine had just `~/Code`, and so on. It's been somewhat messy, and messy in different ways across my three machines. For instance, my laptop had work projects in `~/Code/Job`, my work machine had just `~/Code`, and so forth.
And it's strange that I wasn't able to find any existing solution to that problem. I can't be the only one with that problem, can I? Strangely, I couldn't find and existing solution to that problem. Surely, I can't be the only one facing that issue, can I?
Anyway, I'm lucky to know my way in (make-yourself-a) Swiss Army Knife of computing called [Emacs](https://www.gnu.org/software/emacs/), so... below is my attempt to make something of it. And another entry to add to the already substantial list of my Emacs uses. Fortunately, I'm well-acquainted (make-yourself-a) Swiss Army Knife of computing called [Emacs](https://www.gnu.org/software/emacs/), so... below is my attempt to make something of it. And another addition to the already substantial list of my Emacs uses.
Also, my `M-x magit-log-buffer-file` shows I've created that file on the same day I had written the embedded toot, so this must be the longest Emacs thing I've been figuring out. And it's also probably the least portable, but I nevertheless hope you'll find it useful. Also, my `M-x magit-log-buffer-file` shows I've created that file on the same day I had written the embedded toot, so this must be the longest Emacs thing I've been figuring out. And it's probably the least portable, but I nevertheless hope you find it useful.
## Idea {#idea} ## Idea {#idea}
{{< figure src="/images/index/index.png" >}} {{< figure src="/images/index/index.png" >}}
So, I've decided to try declarative filesystem management. So, I decided to try declarative filesystem management.
At the core, there's my work-in-progress adaptation of [Johnny.Decimal](https://johnnydecimal.com/)[^fn:1]. Essentially, it proposes to prefix your folders with numbers like `12.34`, where: At the core is my work-in-progress adaptation of [Johnny.Decimal](https://johnnydecimal.com/)[^fn:1]. Essentially, it suggests prefixing your folders with numbers like `12.34`, where:
- the first digit is "[category](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)" - the first digit is the "[category](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)";
- the second digit is "[area](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)" - the second digit is the "[area](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)";
- the last two digits are the [ID](https://johnnydecimal.com/10-19-concepts/11-core/11.03-ids/). - the last two digits are the [ID](https://johnnydecimal.com/10-19-concepts/11-core/11.03-ids/).
The point is to organize your folder structure and limit its depth, which should make finding things quicker and more straightforward. Check the website for a more thorough description. The point is to organize your folder structure, limiting its depth for quicker and more straightforward access. Check the website for a more thorough description.
So, what I want is: So, what I want is to:
- to define a Jonny.Decimal-esque file tree in a single [Org](https://orgmode.org/) file; - define a Jonny.Decimal-esque file tree in a single [Org](https://orgmode.org/) file;
- have different nodes of that file tree active on different machines, e.g. I don't want [my Emacs stuff](https://github.com/SqrtMinusOne?tab=repositories&q=&type=&language=emacs+lisp&sort=) on my work machine; - have different nodes of that file tree active on different machines, e.g. I don't want [my Emacs stuff](https://github.com/SqrtMinusOne?tab=repositories&q=&type=&language=emacs+lisp&sort=) on my work machine;
- use different tools to sync different nodes (as of now [git](https://git-scm.com/), [MEGA](https://mega.nz/), and "nothing"). - use different tools to sync different nodes (currently [git](https://git-scm.com/), [MEGA](https://mega.nz/), and "nothing").
### Folder structure {#folder-structure} ### Folder structure {#folder-structure}
As I said, I tried (and still trying) to adapt the proposed scheme to better suit my needs. Here's a subset of my current tree. As I said, I tried (and still trying) to adapt the proposed scheme to better suit my needs. Here's a subset of my current tree:
```text ```text
10-19 Code 10-19 Code
@ -76,11 +76,11 @@ As I said, I tried (and still trying) to adapt the proposed scheme to better sui
33 Library 33 Library
``` ```
The root of the tree is my `$HOME`. The entry at the third (or second) level can be either an entity it itself (such as a git repository), or a "project root". The root of the tree is my `$HOME`. The entry at the third (or second) level can be either an entity itself (such as a git repository), or a "project root".
In several places I use year references (`Y20`) instead of the plain `AC.ID`. This is mainly to group things by academic years, e.g. to find all my publications or students in some year, which I need for occasional reports. I also have semester references (`SEM10`) for my undergraduate studies. In several places, I use year references (`Y20`) instead of the plain `AC.ID`. This is mainly to group things by academic years, e.g. to find all my publications or students in a specific year, which I need for occasional reports. I also have semester references (`SEM10`) for my undergraduate studies.
Project structure is also more or less standard. Johnny.Decimal [proposes](https://johnnydecimal.com/10-19-concepts/13-multiple-projects/13.01-introduction/) to use `PRO.AC.ID` to manage multiple projects, but this doesn't seem to fit quite as well to my case, so I came up with the following: The project structure is more or less standard. Johnny.Decimal [proposes](https://johnnydecimal.com/10-19-concepts/13-multiple-projects/13.01-introduction/) using `PRO.AC.ID` to manage multiple projects, but this doesn't seem to fit quite as well in my case. So I came up with the following:
```text ```text
10.03 Digital Trajectories ; project root 10.03 Digital Trajectories ; project root
@ -101,17 +101,17 @@ Perhaps this is too verbose (`10.03.R.01`), but it works for now.
### Tools choice {#tools-choice} ### Tools choice {#tools-choice}
As I've said, my current options to manage a particular node are: As I mentioned earlier, my current options to manage a particular node are:
- [git](https://git-scm.com/); - [git](https://git-scm.com/);
- [MEGA](https://mega.nz/) - for files that don't fit into git, such as DOCX documents, photos, etc.; - [MEGA](https://mega.nz/) - for files that don't fit into git, such as DOCX documents, photos, etc.;
- nothing - something that I don't need to sync across machines, e.g. database dumps. - "nothing" - for something that I don't need to sync across machines, e.g. database dumps.
One other tool I considered was [restic](https://github.com/restic/restic). It's an interesting backup &amp; sync solution, with built-in encryption, snapshots, etc. Another tool I considered was [restic](https://github.com/restic/restic). It's an interesting backup &amp; sync solution with built-in encryption, snapshots, etc.
My problem is that its repositories are only accessible via restic. So, even if I use something like MEGA as a backend, I won't be able to use the MEGA file-sharing features, which I occasionally want for document or photo folders. So for now I'm more interested in synchronizing the file tree in MEGA with [MEGAcmd](https://github.com/meganz/MEGAcmd) (and also clean up the mess there, two birds with one stone). However, a challenge I encountered is that its repositories are only accessible via restic. So, even if I use something like MEGA as a backend, I won't be able to use the MEGA file-sharing features, which I occasionally want for document or photo folders. Hence, for now, I'm more interested in synchronizing the file tree in MEGA with [MEGAcmd](https://github.com/meganz/MEGAcmd) (and also clean up the mess up there).
Another interesting tool is [rclone](https://rclone.org/), which provides a single interface for multiple services like Google Drive, Dropbox, S3, WebDAV. It also supports MEGA, but requires turning off the two-factor authentication, which I don't want. Another interesting tool is [rclone](https://rclone.org/), which provides a single interface for multiple services like Google Drive, Dropbox, S3, WebDAV. It also supports MEGA, but it requires turning off the two-factor authentication, which I don't want.
## Implementation {#implementation} ## Implementation {#implementation}
@ -132,7 +132,7 @@ And a package called [ini.el](https://github.com/daniel-ness/ini.el) to parse IN
:straight (:host github :repo "daniel-ness/ini.el")) :straight (:host github :repo "daniel-ness/ini.el"))
``` ```
The rest is built-in into Emacs. The rest is built into Emacs.
### Org tree {#org-tree} ### Org tree {#org-tree}
@ -155,8 +155,8 @@ The org tree is located in my `org-mode` folder in a file called `index.org`:
Each "area" is an Org header with the `folder` tag; the Org hierarchy forms the file tree. A header can have the following properties: Each "area" is an Org header with the `folder` tag; the Org hierarchy forms the file tree. A header can have the following properties:
- `machine` - list of hostnames for which the node is active (or `nil`) - `machine` - a list of hostnames for which the node is active (or `nil`)
- `kind` - `mega`, `git` or `dummy` - `kind` - `mega`, `git`, or `dummy`
- `remote` - remote URL for `git` - `remote` - remote URL for `git`
- `symlink` - in case the folder has to be symlinked somewhere else[^fn:2] - `symlink` - in case the folder has to be symlinked somewhere else[^fn:2]
@ -202,13 +202,14 @@ So, let's parse the Org tree. This is done by recursively traversing the tree re
```emacs-lisp ```emacs-lisp
(defun my/index--tree-get-recursive (heading &optional path) (defun my/index--tree-get-recursive (heading &optional path)
"Recursively read index tree from HEADING. "Read the index tree recursively from HEADING.
HEADING is an org-element of type `headline'. HEADING is an org-element of type `headline'.
PATH is the path to the current node. If not provided, it is If PATH is provided, it is the path to the current node. If not
assumed to be the root of the index. The return value is an provided, it is assumed to be the root of the index.
alist, see `my/index--tree-get' for details."
The return value is an alist; see `my/index--tree-get' for details."
(when (eq (org-element-type heading) 'headline) (when (eq (org-element-type heading) 'headline)
(let (val (let (val
(new-path (concat (new-path (concat
@ -241,7 +242,7 @@ alist, see `my/index--tree-get' for details."
val))) val)))
(defun my/index--tree-get () (defun my/index--tree-get ()
"Read index tree from the current org buffer. "Read the index tree from the current org buffer.
The return value is a list of alists, each representing a The return value is a list of alists, each representing a
folder/node. Alists can have the following keys: folder/node. Alists can have the following keys:
@ -268,7 +269,7 @@ folder/node. Alists can have the following keys:
#### Verify tree {#verify-tree} #### Verify tree {#verify-tree}
I also want to make sure that I didn't mess up the numbers, i.e. didn't place `10.02` under `11`, and so on. I also want to make sure that I didn't mess up the numbers, i.e., didn't place `10.02` under `11`, and so on.
To do that, we first need to extract the number from the name: To do that, we first need to extract the number from the name:
@ -357,7 +358,7 @@ Finally, we need to narrow the tree to only leave nodes that are active for the
Next, apply the tree to the filesystem. Next, apply the tree to the filesystem.
I've decided to implement this by generating a bash script and executing it with `bash +x`. This way I can check the required changes in advance and avert potential loss of data if something unexpected happens. I've decided to implement this by generating a bash script and executing it with `bash +x`. This way, I can check the required changes in advance and avert potential data loss if something unexpected happens.
One command for the script will be a list like: One command for the script will be a list like:
@ -378,14 +379,13 @@ FULL-TREE and TREE are forms as defined by `my/index--tree-get'. TREE
is the narrowed FULL-TREE (returned by `my/index--tree-narrow'). is the narrowed FULL-TREE (returned by `my/index--tree-narrow').
ACTIVE-PATHS is a list of paths that are currently active. If not ACTIVE-PATHS is a list of paths that are currently active. If not
provided, it is computed from TREE, i.e. as those paths that have to provided, it is computed from TREE.
exists on the current machine.
The return value is a list of alists with the following keys: The return value is a list of alists with the following keys:
- path - the path of the folder - path - the path of the folder
- exists - whether the folder exists on the filesystem - exists - whether the folder exists on the filesystem
- has-to-exist - whether the folder exists in the tree - has-to-exist - whether the folder exists in the tree
- extra - if the folder exists in the filesystem but not in tree. - extra - if the folder exists in the filesystem but not in the tree.
- children - a list of alists with the same keys for the children of - children - a list of alists with the same keys for the children of
the folder." the folder."
(let ((active-paths (or active-paths (my/index--tree-get-paths tree)))) (let ((active-paths (or active-paths (my/index--tree-get-paths tree))))
@ -587,8 +587,7 @@ To sync git, we just need to clone the required git repos. Removing the repos is
"Get commands to clone the yet uncloned git repos in TREE. "Get commands to clone the yet uncloned git repos in TREE.
TREE is a form a defined by `my/index--tree-get'. This is supposed to TREE is a form a defined by `my/index--tree-get'. This is supposed to
be the tree narrowed to the current machine be the tree narrowed to the current machine (`my/index--tree-narrow').
(`my/index--tree-narrow').
The return value is a list of commands as defined by The return value is a list of commands as defined by
`my/index--commands-display'." `my/index--commands-display'."
@ -866,7 +865,7 @@ for recursive calls.
The result is a list of alists with the following keys: The result is a list of alists with the following keys:
- `:names` - list of names, e.g. - `:names` - list of names, e.g.
(\"10.01 Something\" \"10.01.01 Something\") (\"10.01 Something\" \"10.01.01 Something\")
: `:path` - path to the folder, e.g. - `:path` - path to the folder, e.g.
\"/path/10 stuff/10.01 Something/10.01.01 Something/\" \"/path/10 stuff/10.01 Something/10.01.01 Something/\"
- `:child-navs` - list of child navigation structures (optional)" - `:child-navs` - list of child navigation structures (optional)"
(seq-sort-by (seq-sort-by
@ -925,7 +924,7 @@ The return value is a form as defined by `my/index--nav-get'."
#### Emacs interface {#emacs-interface} #### Emacs interface {#emacs-interface}
As for Emacs interface, a plain `completing-read` is sufficient, except that I don't want [prescient.el](https://github.com/radian-software/prescient.el) to interfere with the default ordering of elements. As for Emacs interface, `completing-read` is sufficient, except that I don't want [prescient.el](https://github.com/radian-software/prescient.el) to interfere with the default ordering of elements.
```emacs-lisp ```emacs-lisp
(defun my/index--nav-prompt (nav) (defun my/index--nav-prompt (nav)
@ -963,7 +962,7 @@ command as follows:
- '(4): Select an indexed directory, and select a child indexed - '(4): Select an indexed directory, and select a child indexed
directory if available. directory if available.
- If in an indexed directory with indexed children (a project): - If in an indexed directory with indexed children (a project):
- nil: Select another indexed directory from the project - nil: Select another indexed directory from the project.
- '(4): Select a top-level indexed directory (the same as nil for - '(4): Select a top-level indexed directory (the same as nil for
the previous case). the previous case).
- '(16): The same as '(4) for the previous case. - '(16): The same as '(4) for the previous case.

2
org/2023-11-11-index.org
View file

@ -4,7 +4,7 @@
#+DATE: 2023-11-11 #+DATE: 2023-11-11
#+HUGO_TAGS: emacs #+HUGO_TAGS: emacs
#+HUGO_TAGS: orgmode #+HUGO_TAGS: orgmode
#+HUGO_DRAFT: true #+HUGO_DRAFT: false
#+begin_abstract #+begin_abstract
The post describes a Johnny.Decimal-inspired filesystem structure, declared in an org file and synchronized across machines. Different folders are available on different machines. The post describes a Johnny.Decimal-inspired filesystem structure, declared in an org file and synchronized across machines. Different folders are available on different machines.