sqrtminusone/elfeed-summary
1
0
Fork 0
mirror of https://github.com/SqrtMinusOne/elfeed-summary.git synced 2025-12-10 17:43:03 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #10 from SqrtMinusOne/auto-tags

Browse source 
Automatic tree generation
This commit is contained in:
Pavel Korytov 2022-12-10 16:49:25 +03:00 • committed by GitHub
commit ccbaf85d9e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 478 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

122
README.org
View file

@ -24,6 +24,8 @@ The tree consists of:
- searches;
- groups, that can include other groups, feeds, and searches.
Groups can also be generated automatically.
Available keybindings in the summary mode:
| Keybinding | Command | Description |
@ -49,6 +51,10 @@ This is a list of these possible items:
Query extracts a subset of elfeed feeds based on the given criteria. Each found feed will be represented as a line.
- Search =(search . <search-params>)=
Elfeed search, as defined by =elfeed-search-set-filter=.
- Tags tree =(auto-tags . <auto-tags-params>)=
A tree generated automatically from the available tags.
- Tag groups =(tag-groups . <tag-group-params>)=
Insert one tag as one group.
- a few special forms
=<group-params>= is an alist with the following keys:
@ -89,6 +95,21 @@ Query examples:
- =:title= (mandatory) title.
- =:tags= - list of tags to get the face of the entry.
=<auto-tags-params>= is an alist with the following keys:
- =:max-level= - maximum level of the tree (default 2)
- =:source= - which feeds to use to build the tree.
Can be =:misc= (default) or =(query . <query-params>)=.
- =:original-order= - do not try to build a more concise tree by
putting the most frequent tags closer to the root of the tree.
- =:faces= - list of faces for groups.
=<tag-group-params>= is an alist with the following keys:
- =:source= - which feeds to use to build the tree.
Can be =:misc= (default) or =(query . <query-params>)=.
- =:repeat-feeds= - allow feeds to repeat. Otherwise, each feed is
assigned to group with the least amount of members.
- =:face= - face for groups.
Available special forms:
- =:misc= - print out feeds, not found by any query above.
@ -150,11 +171,106 @@ Here is an excerpt from my configuration that was used to produce this screensho
(:title . "Ungrouped")
(:elements :misc))))))
#+end_src
** Automatic generation of groups
*** =auto-tags=
As described in the [[*Tree configuration][tree configuration]] section, there are two ways to avoid defining all the relevant groups manually, =auto-tags= and =tag-groups=. Both use tags that are defined in =elfeed-feeds=.
=auto-tags= tries to build the most concise tree from these tags. E.g. if we have feeds:
#+begin_example
feed1 tag1 tag2
feed2 tag1 tag2
feed3 tag1 tag3
feed4 tag1 tag3
#+end_example
It will create the following tree:
- tag1
- tag2
- feed1
- feed2
- tag3
- feed3
- feed4
The tree is truncated by =:max-level=, which is 2 by default.
If tags don't form this kind of hierarchy in =elfeed-feeds=, the algorithm will still try to build the most "optimal" tree, where the most frequent tags are on the top.
To avoid that you can set =(:original-order . t)=, in which case each feed will be placed at the path =tag1 tag2 ... tagN feed=, where the order of tags is the same as in =elfeed-feeds=. By the way, this allows reproducing the hierarchy of [[https://github.com/remyhonig/elfeed-org][elfeed-org]], e.g. this structure:
#+begin_example
,* tag1 :tag1:
,** feed1
,** feed2 :tag2:
,** feed3 :tag2:
,* tag3 :tag3:
,** feed4 :tag2:
,** feed5 :tag2:
,** feed6 :tag2:
#+end_example
Will be converted to this:
- tag1
- feed1
- tag2
- feed2
- feed3
- tag3
- tag2
- feed4
- feed5
- feed6
Whereas without =:original-order= the structure will be:
- tag1
- feed1
- tag2
- tag1
- feed2
- feed3
- tag3
- feed4
- feed5
- feed6
*** =tag-groups=
The second option is =tag-groups=, which creates a group for each tag.
By default, each feed is assigned to its less frequent tag. This can be turned off by setting =(:repeat-feeds . t)=.
E.g., the elfeed-org setup from the section above will be converted to this structure:
- tag1
- feed1
- feed2
- feed3
- tag3
- feed4
- feed5
- feed6
And with =:repeat-feeds=:
- tag1
- feed1
- feed2
- feed3
- tag2
- feed2
- feed3
- feed4
- feed5
- feed6
- tag3
- feed4
- feed5
- feed6
*** Common options
Both =auto-tags= and =tag-groups= allow setting the =:search= parameter.
The default value is =(:search . :misc)=, i.e. use feeds that weren't found by other queries.
Passing =(:search . (query . <query-params>))= is another option.
** Faces
Faces for groups by default use the =elfeed-summary-group-faces= variable, which serves as a list of faces for each level of the tree. Individual group faces can be overridden with the =:face= attribute.
Group faces by default use the =elfeed-summary-group-faces= variable, which serves as a list of faces for each level of the tree. Individual group faces can be overridden with the =:face= attribute.
Faces for feeds by default reuse [[https://github.com/skeeto/elfeed#custom-tag-faces][the existing elfeed mechanism]]. The tags for feeds are taken from the =elfeed-feeds= variable; if a feed has at least one unread entry, the unread tag is added to the list. This can be overridden by setting the =elfeed-summary-feed-face-fn= variable.
Feed faces by default reuse [[https://github.com/skeeto/elfeed#custom-tag-faces][the existing elfeed mechanism]]. The tags for feeds are taken from the =elfeed-feeds= variable; if a feed has at least one unread entry, the unread tag is added to the list. This can be overridden by setting the =elfeed-summary-feed-face-fn= variable.
Searches are mostly the same as feeds, but tags for the search are taken from the =:tags= attribute. This also can be overridden with =elfeed-summary-search-face-fn= variable.
** Opening =elfeed-search= in other window
@ -168,8 +284,6 @@ Then =RET= and =M-RET= in the =elfeed-summary= buffer will open the search buffe
=elfeed-summary-width= regulates the width of the remaining summary window in this case. It is useful because the data in the search buffer is generally wider than in the summary buffer. The variable can also be set to =nil= to disable this behavior.
** Other options
Also take a look at =M-x customize-group elfeed-summary= for the rest of available options.
Setting a unique URL for every feed resolves the problem.
* Ideas and alternatives
The default interface of elfeed is just a list of all entries. Naturally, it gets hard to navigate when there are a lot of sources with varying frequencies of posts.

367
elfeed-summary.el
View file

@ -125,6 +125,44 @@
(repeat symbol))
(cons (const :tag "Add the default filter string" :add-default)
(boolean :tag "Add the default filter string")))))
(cons :tag "Generate a tree from tags"
(const auto-tags)
(repeat :tag "Tree generation parameters"
(choice
(cons :tag "Source"
(const :tag "Source" :source)
(choice
(cons :tag "Query"
(const query)
elfeed-summary-query)
(const :tag "Misc feeds" :misc)))
(cons :tag "Maximum tree level"
(const :tag "Maximum tree level" :max-level)
(number :tag "Value"))
(cons :tag "Original tag order"
(const :tag "Original tag order" :original-order)
(boolean :tag "Original tag order"))
(cons :tag "Faces"
(const :tag "Faces" :faces)
(repeat
(face :tag "Face"))))))
(cons :tag "Insert each tag as group"
(const tag-groups)
(repeat :tag "Tag group parameters"
(choice
(cons :tag "Source"
(const :tag "Source" :source)
(choice
(cons :tag "Query"
(const query)
elfeed-summary-query)
(const :tag "Misc feeds" :misc)))
(cons :tag "Allow feeds to repeat"
(const :tag "Allow feeds to repeat" :repeat-feeds)
(boolean :tag "Allow feeds to repeat"))
(cons :tag "Face"
(const :tag "Face" :faces)
(face :tag "Face")))))
(const :tag "Misc feeds" :misc))))
(defgroup elfeed-summary ()
@ -151,6 +189,10 @@ This is a list of these possible items:
Each found feed will be represented as a line.
- Search `(search . <search-params>)'
Elfeed search, as defined by `elfeed-search-set-filter'.
- Tags tree `(auto-tags . <auto-tags-params>)'
A tree generated automatically from the available tags.
- Tag groups `(tag-groups . <tag-group-params>)'
Insert one tag as one group.
- a few special forms
`<group-params>' is an alist with the following keys:
@ -174,8 +216,6 @@ This is a list of these possible items:
- `(or <q-1> <q-2> ... <q-n>)' or `(<q-1> <q-2> ... <q-n>)'
Match if any of the conditions 1, 2, ..., n match.
- `(not <query>)'
Feed tags for the query are determined by the `elfeed-feeds'
variable.
Query examples:
@ -195,6 +235,21 @@ Query examples:
- `:add-default' - if t, prepend the filter with
`elfeed-summary-default-filter'.
`<auto-tags-params>' is an alist with the following keys:
- `:max-level' - maximum level of the tree (default 2)
- `:source' - which feeds to use to build the tree.
Can be `:misc' (default) or `(query . <query-params>)'.
- `:original-order' - do not try to build a more concise tree by
putting the most frequent tags closer to the root of the tree.
- `:faces' - list of faces for groups.
`<tag-group-params>' is an alist with the following keys:
- `:source' - which feeds to use to build the tree.
Can be `:misc' (default) or `(query . <query-params>)'.
- `:repeat-feeds' - allow feeds to repeat. Otherwise, each feed is
assigned to group with the least amount of members.
- `:face' - face for groups.
Available special forms:
- `:misc' - print out feeds, not found by any query above.
@ -270,6 +325,18 @@ ordering."
:group 'elfeed-summary
:type 'function)
(defcustom elfeed-summary-auto-tags-group-title-fn
#'elfeed-summary--auto-tags-group-title
"Function to get the title of an autogenerated group.
Accepts the only parameter, which is a tree node created by
`elfeed-summary--arrange-sequences-in-tree'.
See `elfeed-summary--auto-tags-group-title' for the default
implementation."
:group 'elfeed-summary
:type 'function)
(defcustom elfeed-summary-refresh-on-each-update nil
"Whether to refresh the elfeed summary buffer after each update.
@ -523,6 +590,262 @@ Implented the same way as `elfeed-search--update-list'."
(unread-ids . ,unread-ids)
(total . ,total)))))
(defun elfeed-summary--get-tags-ordered ()
"Return the list of elfeed tags, properly ordered.
The tags are ordered (1) by their most frequent position in
`elfeed-feeds' and (2) alphabetically."
(let* ((tags-order
;; list of (tag . ((<position-1> . <freq-1>) (<position-2> . <freq-2>) ...))
(cl-loop with tags-order = '()
for feed in elfeed-feeds
do (cl-loop for tag in (cdr feed)
for i from 0
unless (alist-get tag tags-order)
do (push (list tag) tags-order)
do (cl-incf (alist-get
i (alist-get tag tags-order) 0)))
finally return tags-order))
;; list of (tag . <most-frequent-position>)
(tags-most-freq-order
(cl-loop for (tag . order) in tags-order collect
(cons
tag
(car
(cl-reduce
(lambda (acc value)
(if (> (cdr value) (cdr acc))
value
acc))
order
:initial-value '(-1 . -1)))))))
(mapcar
#'car
(seq-sort
(lambda (datum1 datum2)
(if (not (= (cdr datum1) (cdr datum2)))
(< (cdr datum1) (cdr datum2))
(string-lessp (symbol-name (car datum1))
(symbol-name (car datum2)))))
tags-most-freq-order))))
(defun elfeed-summary--build-tree-auto-tags-reorder-tags (feeds)
"Reorder tags in FEEDS.
FEEDS is a list of (<feed> . <tags>), where <feed> is an instance of
`elfeed-feed' and <tags> is a list of tag symbols."
(let* ((all-tags (elfeed-summary--get-tags-ordered))
(tag-priority (make-hash-table)))
(cl-loop for tag in all-tags
for i from 0
do (puthash tag i tag-priority))
(cl-loop for (feed . tags) in feeds
collect
(cons feed
(seq-sort-by (lambda (tag) (gethash tag tag-priority))
#'> tags)))))
(defun elfeed-summary--compare-sequences (sequence1 sequence2)
"Compare SEQUENCE1 and SEQUENCE2.
Both are lists of symbols."
(cond
((null sequence1) t)
((null sequence2) nil)
(t (let ((item1 (symbol-name (car sequence1)))
(item2 (symbol-name (car sequence2))))
(if (string-equal item1 item2)
(elfeed-summary--compare-sequences (cdr sequence1)
(cdr sequence2))
(string-lessp item1 item2))))))
(defun elfeed-summary--arrange-sequences-in-tree (sequences)
"Arrange SEQUENCES in a tree structure.
Each element of SEQUENCES is a list of symbols.
The resulting structure is an alist of tree nodes with the following keys:
- `value' - the current node symbol
- `children' - child nodes
- `sequences' - sequences at this node
The root of the tree has the value of nil."
(let ((ordered-sequences
(seq-reverse
(seq-sort #'elfeed-summary--compare-sequences sequences)))
(tree `(,nil . ((value . ,nil) (children . ,nil) (sequences . ,nil))))
current-tree-pos
(processed-sequences (make-hash-table :test #'equal)))
(dolist (sequence ordered-sequences)
(unless (gethash sequence processed-sequences)
(setq current-tree-pos tree)
(dolist (value sequence)
(if-let ((value-in-tree (alist-get value (alist-get 'children current-tree-pos))))
(setq current-tree-pos value-in-tree)
(setq current-tree-pos
(setf
(alist-get value (alist-get 'children current-tree-pos))
`((value . ,value) (children . ,nil) (sequences . ,nil))))))
(push sequence (alist-get 'sequences current-tree-pos))
(puthash sequence t processed-sequences)))
tree))
(defun elfeed-summary--auto-tags-group-title (child-tree)
"Default function to get the name of an auto-tags group.
CHILD-TREE is a structure as defined in
`elfeed-summary--arrange-sequences-in-tree', with tag lists as
sequences."
(symbol-name (alist-get 'value child-tree)))
(defun elfeed-summary--build-tree-auto-tags-recursive
(param tree feeds-by-tag-sequence unread-count total-count &optional level)
"Recursively create the auto-tags tree.
PARAM is an `<auto-tags-params>' form as described in
`elfeed-summary-settings'. TREE is the result of applying
`elfeed-summary--arrange-sequences-in-tree' onto the list of tags of
all feeds.
FEEDS-BY-TAG-SEQUENCE is a hashmap with lists of tags as keys and
instances of `elfeed-feed' as values. This is used to figure out
feeds in a particular TREE node.
UNREAD-COUNT and TOTAL-COUNT are hashmaps with feed ids as keys and
corresponding numbers of entries as values.
LEVEL is the current level of recursion, which is 0 by default."
(unless level
(setq level 0))
(let ((max-level (or (alist-get :max-level (cdr param)) 2))
(face (when-let (faces (alist-get :faces (cdr param)))
(nth (% level (length faces)) faces))))
(append
;; Just append all the feeds at the current level
(cl-loop for sequence in (alist-get 'sequences tree) append
(cl-loop for feed in (seq-sort
#'elfeed-summary--feed-sort-fn
(gethash sequence feeds-by-tag-sequence))
collect (elfeed-summary--build-tree-feed
feed unread-count total-count)))
;; Go deeper if we can
(when (< level max-level)
(cl-loop
for datum in (alist-get 'children tree)
for child-tree = (cdr datum) collect
`(group . ((params . ((:title
. ,(funcall elfeed-summary-auto-tags-group-title-fn
child-tree))))
(face . ,face)
(children . ,(elfeed-summary--build-tree-auto-tags-recursive
param child-tree feeds-by-tag-sequence
unread-count total-count (1+ level)))))))
;; If we can't go deeper, this will just append all the feeds to
;; the current level anyway
(when (>= level max-level)
(cl-loop for datum in (alist-get 'children tree) append
(elfeed-summary--build-tree-auto-tags-recursive
param (cdr datum) feeds-by-tag-sequence
unread-count total-count (1+ level)))))))
(defun elfeed-summary--build-tree-get-feeds (param misc-feeds)
"Get feeds for PARAM.
PARAM is an alist with the optional `:source' key. The value can be
either `(query . <query-params>)' or `:misc' (default).
MISC-FEEDS is the list of feeds used for `:misc'.
The result is a list of items like (`<feed>' tag1 tag2 ...), where
`<feed>' is an instance of `elfeed-feed'."
(let* ((source (alist-get :source (cdr param))))
(mapcar
(lambda (feed)
(cons feed (alist-get (elfeed-feed-id feed) elfeed-feeds
nil nil #'equal)))
(cond ((or (eq source :misc) (null source))
misc-feeds)
((and (listp source) (eq (car source) 'query))
(elfeed-summary--get-feeds (cdr source)))
(t (error "Invalid source: %s" source))))))
(defun elfeed-summary--build-tree-auto-tags (param unread-count total-count misc-feeds)
"Create the auto-tags tree.
PARAM is a cons cell `(auto-tags . <auto-tags-params>)', where
`<auto-tags-params>' is described in `elfeed-summary-settings'.
UNREAD-COUNT and TOTAL-COUNT are hashmaps with feed ids as keys and
corresponding numbers of entries as values.
MISC-FEEDS is a list of feeds that were not used in PARAMS."
(let ((feeds (elfeed-summary--build-tree-get-feeds param misc-feeds))
(reorder-tags (not (alist-get :original-order (cdr param)))))
(when reorder-tags
(setq feeds (elfeed-summary--build-tree-auto-tags-reorder-tags feeds)))
(let ((tree (elfeed-summary--arrange-sequences-in-tree
(mapcar #'cdr feeds)))
(feeds-by-tag-sequence (make-hash-table :test #'equal)))
(cl-loop
for (feed . sequence) in feeds
do (puthash sequence (cons feed (gethash sequence feeds-by-tag-sequence))
feeds-by-tag-sequence))
(elfeed-summary--build-tree-auto-tags-recursive
param tree feeds-by-tag-sequence unread-count total-count))))
(defun elfeed-summary--build-tree-tag-groups (param unread-count total-count misc-feeds)
"Create the tag-groups tree.
PARAM is a cell of `(tag-groups . <tag-group-params>)', with the
`<tag-group-params>' form as defined in `elfeed-summary-settings'.
UNREAD-COUNT and TOTAL-COUNT are hashmaps with feed ids as keys and
corresponding numbers of entries as values.
MISC-FEEDS is a list of feeds that were not used in PARAMS."
(let ((feeds (elfeed-summary--build-tree-get-feeds param misc-feeds))
(repeat-feeds (alist-get :repeat-feeds (cdr param)))
(face (alist-get :face (cdr param)))
(groups (make-hash-table)))
(if (not repeat-feeds)
(let ((tag-freqs (make-hash-table)))
(cl-loop for feed in feeds do
(cl-loop for tag in (cdr feed) do
(puthash
tag (1+ (gethash tag tag-freqs 0))
tag-freqs)))
(cl-loop for feed in feeds
for min-freq-tag = (cl-reduce
(lambda (acc tag)
(let ((freq (gethash tag tag-freqs)))
(if (or (null (cdr acc)) (< freq (cdr acc)))
(cons tag freq)
acc)))
(cdr feed)
:initial-value '(nil . nil))
when min-freq-tag do
(puthash (car min-freq-tag)
(cons (car feed) (gethash min-freq-tag groups))
groups)))
(cl-loop for feed in feeds do
(cl-loop for tag in (cdr feed) do
(puthash tag (cons (car feed) (gethash tag groups)) groups))))
(let ((groups-list (seq-sort-by
(lambda (f) (symbol-name (car f)))
#'string-lessp
(cl-loop for tag being the hash-keys of groups
using (hash-values feeds)
collect (cons tag feeds)))))
(cl-loop for (tag . feeds) in groups-list
collect `(group
. ((params . ((:title . ,(symbol-name tag))))
(face . ,face)
(children . ,(mapcar
(lambda (feed) (elfeed-summary--build-tree-feed
feed unread-count total-count))
(seq-sort
#'elfeed-summary--feed-sort-fn feeds)))))))))
(defun elfeed-summary--build-tree (params unread-count total-count misc-feeds)
"Recursively create the summary details tree.
@ -547,8 +870,15 @@ The resulting form is described in `elfeed-summary--get-data'."
append (cl-loop for feed in (elfeed-summary--get-feeds (cdr param))
collect (elfeed-summary--build-tree-feed
feed unread-count total-count))
else if (and (listp param) (eq (car param) 'auto-tags))
append (elfeed-summary--build-tree-auto-tags
param unread-count total-count misc-feeds)
else if (and (listp param) (eq (car param) 'tag-groups))
append (elfeed-summary--build-tree-tag-groups
param unread-count total-count misc-feeds)
else if (eq param :misc)
append (cl-loop for feed in misc-feeds
append (cl-loop for feed in (seq-sort #'elfeed-summary--feed-sort-fn
misc-feeds)
collect (elfeed-summary--build-tree-feed
feed unread-count total-count))
else do (error "Can't parse: %s" (prin1-to-string param))))
@ -562,7 +892,13 @@ PARAMS is a form as described in `elfeed-summary-settings'."
append (elfeed-summary--extract-feeds
(cdr (assoc :elements (cdr param))))
else if (and (listp param) (eq (car param) 'query))
append (elfeed-summary--get-feeds (cdr param))))
append (elfeed-summary--get-feeds (cdr param))
else if (and (listp param)
(or (eq (car param) 'auto-tags) (eq (car param) 'tag-groups))
(eq (car-safe (alist-get :source (cdr param)))
'query))
append (elfeed-summary--get-feeds
(cdr (alist-get :source (cdr param))))))
(defun elfeed-summary--ensure ()
"Ensure that elfeed database is loaded and feeds are set up."
@ -648,6 +984,20 @@ The return value is a list of alists of the following elements:
(defvar elfeed-summary--search-mark-read nil
"If t, mark the feed as read instead of switching to it.")
(defun elfeed-summary--magit-section-toggle-workaround (section)
"`magit-section-toggle' with a workaround for invisible lines.
SECTION is an instance of `magit-section'.
No idea what I'm doing wrong, but this seems to help."
(interactive (list (save-excursion
(let ((lines (count-lines (point-min) (point-max))))
(while (and (invisible-p (point))
(< (line-number-at-pos) lines))
(forward-line 1)))
(magit-current-section))))
(magit-section-toggle section))
(defvar elfeed-summary-mode-map
(let ((map (make-sparse-keymap)))
(set-keymap-parent map magit-section-mode-map)
@ -658,9 +1008,10 @@ The return value is a list of alists of the following elements:
(define-key map (kbd "R") #'elfeed-summary-update)
(define-key map (kbd "u") #'elfeed-summary-toggle-only-unread)
(define-key map (kbd "U") #'elfeed-summary--action-mark-read)
(define-key map (kbd "<tab>") #'elfeed-summary--magit-section-toggle-workaround)
(when (fboundp #'evil-define-key*)
(evil-define-key* 'normal map
(kbd "<tab>") #'magit-section-toggle
(kbd "<tab>") #'elfeed-summary--magit-section-toggle-workaround
"r" #'elfeed-summary--refresh
"R" #'elfeed-summary-update
"u" #'elfeed-summary-toggle-only-unread
@ -844,12 +1195,14 @@ descent."
(dolist (id ids)
(puthash id t ids-hash))
(with-elfeed-db-visit (entry feed)
;; XXX to shut up the byte compiler
(ignore feed)
(when (and
(gethash (elfeed-entry-id entry) ids-hash nil)
(member elfeed-summary-unread-tag (elfeed-entry-tags entry)))
(setf (elfeed-entry-tags entry)
(seq-filter (lambda (tag) (not (eq elfeed-summary-unread-tag tag)))
(elfeed-entry-tags entry))))) )
(elfeed-entry-tags entry))))))
(elfeed-summary--refresh)))
(defun elfeed-summary--search-notify (widget &rest _)
@ -1103,7 +1456,7 @@ summary buffer."
"Update all the feeds in `elfeed-feeds' and the summary buffer."
(interactive)
(elfeed-log 'info "Elfeed update: %s"
(format-time-string "%B %e %Y %H:%M:%S %Z"))
(format-time-string "%B %e %Y %T %Z"))
;; XXX Here's a remarkably dirty solution. This command is meant to
;; refresh the elfeed-summary buffer after all the feeds have been
;; updated. But elfeed doesn't seem to provide anything to hook