index: remove draft

This commit is contained in:
Pavel Korytov 2023-11-11 13:02:10 +03:00
parent e10cb2ee55
commit 3ee4e2ebf8
2 changed files with 41 additions and 42 deletions

View file

@ -1,9 +1,9 @@
+++
title = "Declarative filesystem management with Emacs & Org Mode"
author = ["Pavel Korytov"]
date = 2023-11-10
date = 2023-11-11
tags = ["emacs", "orgmode"]
draft = true
draft = false
+++
<div class="abstract">
@ -21,39 +21,39 @@ My filesystem is, shall we say, not the most orderly place.
<iframe src="https://emacs.ch/@sqrtminusone/110514686718545191/embed" class="mastodon-embed" style="max-width: 100%; border: 0" width="500" allowfullscreen="allowfullscreen"></iframe><script src="https://emacs.ch/embed.js" async="async"></script>
</center>
It's been kinda messy, and messy in different ways across my three machines. For instance, my laptop had work projects in `~/Code/Job`, my work machine had just `~/Code`, and so on.
It's been somewhat messy, and messy in different ways across my three machines. For instance, my laptop had work projects in `~/Code/Job`, my work machine had just `~/Code`, and so forth.
And it's strange that I wasn't able to find any existing solution to that problem. I can't be the only one with that problem, can I?
Strangely, I couldn't find and existing solution to that problem. Surely, I can't be the only one facing that issue, can I?
Anyway, I'm lucky to know my way in (make-yourself-a) Swiss Army Knife of computing called [Emacs](https://www.gnu.org/software/emacs/), so... below is my attempt to make something of it. And another entry to add to the already substantial list of my Emacs uses.
Fortunately, I'm well-acquainted (make-yourself-a) Swiss Army Knife of computing called [Emacs](https://www.gnu.org/software/emacs/), so... below is my attempt to make something of it. And another addition to the already substantial list of my Emacs uses.
Also, my `M-x magit-log-buffer-file` shows I've created that file on the same day I had written the embedded toot, so this must be the longest Emacs thing I've been figuring out. And it's also probably the least portable, but I nevertheless hope you'll find it useful.
Also, my `M-x magit-log-buffer-file` shows I've created that file on the same day I had written the embedded toot, so this must be the longest Emacs thing I've been figuring out. And it's probably the least portable, but I nevertheless hope you find it useful.
## Idea {#idea}
{{< figure src="/images/index/index.png" >}}
So, I've decided to try declarative filesystem management.
So, I decided to try declarative filesystem management.
At the core, there's my work-in-progress adaptation of [Johnny.Decimal](https://johnnydecimal.com/)[^fn:1]. Essentially, it proposes to prefix your folders with numbers like `12.34`, where:
At the core is my work-in-progress adaptation of [Johnny.Decimal](https://johnnydecimal.com/)[^fn:1]. Essentially, it suggests prefixing your folders with numbers like `12.34`, where:
- the first digit is "[category](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)"
- the second digit is "[area](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)"
- the first digit is the "[category](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)";
- the second digit is the "[area](https://johnnydecimal.com/10-19-concepts/11-core/11.02-areas-and-categories/)";
- the last two digits are the [ID](https://johnnydecimal.com/10-19-concepts/11-core/11.03-ids/).
The point is to organize your folder structure and limit its depth, which should make finding things quicker and more straightforward. Check the website for a more thorough description.
The point is to organize your folder structure, limiting its depth for quicker and more straightforward access. Check the website for a more thorough description.
So, what I want is:
So, what I want is to:
- to define a Jonny.Decimal-esque file tree in a single [Org](https://orgmode.org/) file;
- define a Jonny.Decimal-esque file tree in a single [Org](https://orgmode.org/) file;
- have different nodes of that file tree active on different machines, e.g. I don't want [my Emacs stuff](https://github.com/SqrtMinusOne?tab=repositories&q=&type=&language=emacs+lisp&sort=) on my work machine;
- use different tools to sync different nodes (as of now [git](https://git-scm.com/), [MEGA](https://mega.nz/), and "nothing").
- use different tools to sync different nodes (currently [git](https://git-scm.com/), [MEGA](https://mega.nz/), and "nothing").
### Folder structure {#folder-structure}
As I said, I tried (and still trying) to adapt the proposed scheme to better suit my needs. Here's a subset of my current tree.
As I said, I tried (and still trying) to adapt the proposed scheme to better suit my needs. Here's a subset of my current tree:
```text
10-19 Code
@ -76,11 +76,11 @@ As I said, I tried (and still trying) to adapt the proposed scheme to better sui
33 Library
```
The root of the tree is my `$HOME`. The entry at the third (or second) level can be either an entity it itself (such as a git repository), or a "project root".
The root of the tree is my `$HOME`. The entry at the third (or second) level can be either an entity itself (such as a git repository), or a "project root".
In several places I use year references (`Y20`) instead of the plain `AC.ID`. This is mainly to group things by academic years, e.g. to find all my publications or students in some year, which I need for occasional reports. I also have semester references (`SEM10`) for my undergraduate studies.
In several places, I use year references (`Y20`) instead of the plain `AC.ID`. This is mainly to group things by academic years, e.g. to find all my publications or students in a specific year, which I need for occasional reports. I also have semester references (`SEM10`) for my undergraduate studies.
Project structure is also more or less standard. Johnny.Decimal [proposes](https://johnnydecimal.com/10-19-concepts/13-multiple-projects/13.01-introduction/) to use `PRO.AC.ID` to manage multiple projects, but this doesn't seem to fit quite as well to my case, so I came up with the following:
The project structure is more or less standard. Johnny.Decimal [proposes](https://johnnydecimal.com/10-19-concepts/13-multiple-projects/13.01-introduction/) using `PRO.AC.ID` to manage multiple projects, but this doesn't seem to fit quite as well in my case. So I came up with the following:
```text
10.03 Digital Trajectories ; project root
@ -101,17 +101,17 @@ Perhaps this is too verbose (`10.03.R.01`), but it works for now.
### Tools choice {#tools-choice}
As I've said, my current options to manage a particular node are:
As I mentioned earlier, my current options to manage a particular node are:
- [git](https://git-scm.com/);
- [MEGA](https://mega.nz/) - for files that don't fit into git, such as DOCX documents, photos, etc.;
- nothing - something that I don't need to sync across machines, e.g. database dumps.
- "nothing" - for something that I don't need to sync across machines, e.g. database dumps.
One other tool I considered was [restic](https://github.com/restic/restic). It's an interesting backup &amp; sync solution, with built-in encryption, snapshots, etc.
Another tool I considered was [restic](https://github.com/restic/restic). It's an interesting backup &amp; sync solution with built-in encryption, snapshots, etc.
My problem is that its repositories are only accessible via restic. So, even if I use something like MEGA as a backend, I won't be able to use the MEGA file-sharing features, which I occasionally want for document or photo folders. So for now I'm more interested in synchronizing the file tree in MEGA with [MEGAcmd](https://github.com/meganz/MEGAcmd) (and also clean up the mess there, two birds with one stone).
However, a challenge I encountered is that its repositories are only accessible via restic. So, even if I use something like MEGA as a backend, I won't be able to use the MEGA file-sharing features, which I occasionally want for document or photo folders. Hence, for now, I'm more interested in synchronizing the file tree in MEGA with [MEGAcmd](https://github.com/meganz/MEGAcmd) (and also clean up the mess up there).
Another interesting tool is [rclone](https://rclone.org/), which provides a single interface for multiple services like Google Drive, Dropbox, S3, WebDAV. It also supports MEGA, but requires turning off the two-factor authentication, which I don't want.
Another interesting tool is [rclone](https://rclone.org/), which provides a single interface for multiple services like Google Drive, Dropbox, S3, WebDAV. It also supports MEGA, but it requires turning off the two-factor authentication, which I don't want.
## Implementation {#implementation}
@ -132,7 +132,7 @@ And a package called [ini.el](https://github.com/daniel-ness/ini.el) to parse IN
:straight (:host github :repo "daniel-ness/ini.el"))
```
The rest is built-in into Emacs.
The rest is built into Emacs.
### Org tree {#org-tree}
@ -155,8 +155,8 @@ The org tree is located in my `org-mode` folder in a file called `index.org`:
Each "area" is an Org header with the `folder` tag; the Org hierarchy forms the file tree. A header can have the following properties:
- `machine` - list of hostnames for which the node is active (or `nil`)
- `kind` - `mega`, `git` or `dummy`
- `machine` - a list of hostnames for which the node is active (or `nil`)
- `kind` - `mega`, `git`, or `dummy`
- `remote` - remote URL for `git`
- `symlink` - in case the folder has to be symlinked somewhere else[^fn:2]
@ -202,13 +202,14 @@ So, let's parse the Org tree. This is done by recursively traversing the tree re
```emacs-lisp
(defun my/index--tree-get-recursive (heading &optional path)
"Recursively read index tree from HEADING.
"Read the index tree recursively from HEADING.
HEADING is an org-element of type `headline'.
PATH is the path to the current node. If not provided, it is
assumed to be the root of the index. The return value is an
alist, see `my/index--tree-get' for details."
If PATH is provided, it is the path to the current node. If not
provided, it is assumed to be the root of the index.
The return value is an alist; see `my/index--tree-get' for details."
(when (eq (org-element-type heading) 'headline)
(let (val
(new-path (concat
@ -241,7 +242,7 @@ alist, see `my/index--tree-get' for details."
val)))
(defun my/index--tree-get ()
"Read index tree from the current org buffer.
"Read the index tree from the current org buffer.
The return value is a list of alists, each representing a
folder/node. Alists can have the following keys:
@ -268,7 +269,7 @@ folder/node. Alists can have the following keys:
#### Verify tree {#verify-tree}
I also want to make sure that I didn't mess up the numbers, i.e. didn't place `10.02` under `11`, and so on.
I also want to make sure that I didn't mess up the numbers, i.e., didn't place `10.02` under `11`, and so on.
To do that, we first need to extract the number from the name:
@ -357,7 +358,7 @@ Finally, we need to narrow the tree to only leave nodes that are active for the
Next, apply the tree to the filesystem.
I've decided to implement this by generating a bash script and executing it with `bash +x`. This way I can check the required changes in advance and avert potential loss of data if something unexpected happens.
I've decided to implement this by generating a bash script and executing it with `bash +x`. This way, I can check the required changes in advance and avert potential data loss if something unexpected happens.
One command for the script will be a list like:
@ -378,14 +379,13 @@ FULL-TREE and TREE are forms as defined by `my/index--tree-get'. TREE
is the narrowed FULL-TREE (returned by `my/index--tree-narrow').
ACTIVE-PATHS is a list of paths that are currently active. If not
provided, it is computed from TREE, i.e. as those paths that have to
exists on the current machine.
provided, it is computed from TREE.
The return value is a list of alists with the following keys:
- path - the path of the folder
- exists - whether the folder exists on the filesystem
- has-to-exist - whether the folder exists in the tree
- extra - if the folder exists in the filesystem but not in tree.
- extra - if the folder exists in the filesystem but not in the tree.
- children - a list of alists with the same keys for the children of
the folder."
(let ((active-paths (or active-paths (my/index--tree-get-paths tree))))
@ -587,8 +587,7 @@ To sync git, we just need to clone the required git repos. Removing the repos is
"Get commands to clone the yet uncloned git repos in TREE.
TREE is a form a defined by `my/index--tree-get'. This is supposed to
be the tree narrowed to the current machine
(`my/index--tree-narrow').
be the tree narrowed to the current machine (`my/index--tree-narrow').
The return value is a list of commands as defined by
`my/index--commands-display'."
@ -866,7 +865,7 @@ for recursive calls.
The result is a list of alists with the following keys:
- `:names` - list of names, e.g.
(\"10.01 Something\" \"10.01.01 Something\")
: `:path` - path to the folder, e.g.
- `:path` - path to the folder, e.g.
\"/path/10 stuff/10.01 Something/10.01.01 Something/\"
- `:child-navs` - list of child navigation structures (optional)"
(seq-sort-by
@ -925,7 +924,7 @@ The return value is a form as defined by `my/index--nav-get'."
#### Emacs interface {#emacs-interface}
As for Emacs interface, a plain `completing-read` is sufficient, except that I don't want [prescient.el](https://github.com/radian-software/prescient.el) to interfere with the default ordering of elements.
As for Emacs interface, `completing-read` is sufficient, except that I don't want [prescient.el](https://github.com/radian-software/prescient.el) to interfere with the default ordering of elements.
```emacs-lisp
(defun my/index--nav-prompt (nav)
@ -963,7 +962,7 @@ command as follows:
- '(4): Select an indexed directory, and select a child indexed
directory if available.
- If in an indexed directory with indexed children (a project):
- nil: Select another indexed directory from the project
- nil: Select another indexed directory from the project.
- '(4): Select a top-level indexed directory (the same as nil for
the previous case).
- '(16): The same as '(4) for the previous case.

View file

@ -4,7 +4,7 @@
#+DATE: 2023-11-11
#+HUGO_TAGS: emacs
#+HUGO_TAGS: orgmode
#+HUGO_DRAFT: true
#+HUGO_DRAFT: false
#+begin_abstract
The post describes a Johnny.Decimal-inspired filesystem structure, declared in an org file and synchronized across machines. Different folders are available on different machines.