feat(emms): the first version

org/2021-09-03-emms.org

@@ -0,0 +1,358 @@
+#+HUGO_SECTION: posts
+#+HUGO_BASE_DIR: ../
+#+TITLE: My EMMS and elfeed setup
+#+DATE: 2021-09-03
+#+HUGO_DRAFT: true
+#+HUGO_TAGS: emacs
+#+HUGO_TAGS: emms
+#+HUGO_TAGS: elfeed
+
+* Intro
+[[./images/emms/screenshot.png]]
+
+This is the current state of my quest to live in Emacs, at least in part of reading RSS and music.
+
+Ever since I lost my mind about customizing obscure keyboard-driven software, I tried Inoreader, self-hosted FreshRSS and then newsboat from the RSS side and ncmpcpp+MPD from the audio player side. Naturally, I got curious about whether I can do the same in Emacs.
+
+The respective emacs packages, elfeed and EMMS, proved somewhat tricky to set up, i.e. I had to figure out the source code of both packages. I even submitted a small patch to EMMS to make it parse my MPD library correctly.
+
+But in the end, only extensive customization capacities of Emacs enabled me to make a setup where these parts nicely come together and do more or less exactly what I want. However, this means there are a lot of degrees of freedom involved, so I'll try to cover the important parts and link to the original sources whereever possible.
+
+I'd call it "workflow", but the "work" part does not quite catch the point here.
+
+* MPD
+So, we have to start somewhere.
+
+[[https://www.musicpd.org/][MPD]] is a server for playing music, although it is usually hosted on the local machine, i.e. the one on which you intend to listen to music. There is [[https://www.musicpd.org/clients/][bunch of clients]] available (take a look at [[https://github.com/ncmpcpp/ncmpcpp][ncmpcpp]] is you like terminal-based apps), but here our point of interest is its integration with EMMS.
+
+MPD configuration is a pretty easy process. First, install mpd and [[https://www.musicpd.org/clients/mpc/][mpc]] (a minimal MPD CLI client) from your distribution's package repository. After doing that, you'd have to create a config file at the location =~/.config/mpd/mpd.conf=. Mine looks something like this:
+
+#+begin_src conf-space
+music_directory     "~/Music"
+playlist_directory  "~/.mpd/playlists"
+db_file             "~/.mpd/database"
+log_file            "~/.mpd/log"
+pid_file            "~/.mpd/pid"
+state_file          "~/.mpd/state"
+sticker_file        "~/.mpd/sticker.sql"
+
+audio_output {
+  type    "pulse"
+  name    "My Pulse Output"
+}
+#+end_src
+
+Here =music_directory= is, well, a directory in which MPD will look for music files. Take a look at [[https://linux.die.net/man/5/mpd.conf][man mpd.conf]] and [[https://github.com/MusicPlayerDaemon/MPD/blob/master/doc/mpdconf.example][the default config example]] for more information.
+
+Because MPD is a daemon, it has to be started in order to work. The easiest way is to add =mpd= to your init system, e.g. with GNU Shepherd:
+
+#+begin_src scheme
+(define mpd
+  (make <service>
+    #:provides '(mpd)
+    #:respawn? #t
+    #:start (make-forkexec-constructor '("mpd" "--no-daemon"))
+    #:stop (make-kill-destructor)))
+#+end_src
+
+You can also launch =mpd= manually, as it will daemonize itself by default. To check if MPD is working, run =mpc status=:
+#+begin_src bash
+mpc status
+#+end_src
+
+#+RESULTS:
+: volume: n/a   repeat: off   random: off   single: off   consume: off
+
+Take a look at [[https://mpd.readthedocs.io/en/stable/user.html#configuration][the official documentation]] for more information on this subject.
+
+** =music_directory=
+The next question after we've set up MPD is how to organize the music directory.
+
+MPD itself is not too concerned about the structure of your =music_directory=, because it uses audio tags to classify the files. However, if you want to have album covers in EMMS, you need to have one folder per one album. Otherwise and in other respects the structure can be arbitrary.
+
+But that means we need to tag the audio files. My favorite audio-tagging software is [[https://picard.musicbrainz.org/][MusicBrainz Picard]], which can set tags automatially even if the file has no metadata at all, and move the file automatically according to the configuration. The aforementioned ncmpcpp also has a decent tag editor; finally there is a simple [[https://mutagen.readthedocs.io/en/latest/man/mid3v2.html][mutagen-based CLI tool]].
+
+* EMMS and MPD
+[[https://www.gnu.org/software/emms/][EMMS]] is the Emacs Multimedia System, which can get play things from various sources using various players. It is a part of Emacs, which means you can use the built-in version, but I advice to use the more recent git version, which has a few useful patches.
+
+Install it however you usually install packages in Emacs; I use use-package + straight:
+
+#+begin_src emacs-lisp
+(use-package emms
+  :straight t)
+#+end_src
+
+** Setup & MPD integration
+Now we have to setup EMMS. The following expressions have to be executed after EMMS is loaded, which means we can add them to the =:config= section of the =use-package= expression above.
+
+First, EMMS exposes a handy function which loads all the stable EMMS features. You can take a look at its source and pick the features you need or load everything:
+#+begin_src emacs-lisp
+(require 'emms-setup)
+(emms-all)
+#+end_src
+
+Then we need to set up a directory for EMMS files and the required parameters for =emms-player-mpd=. Note that =emms-player-mpd-music-directory= should be set to the same value as =music_directory= in =mpd.conf=.
+
+#+begin_src emacs-lisp
+(setq emms-source-file-default-directory (expand-file-name "~/Music/"))
+
+(setq emms-player-mpd-server-name "localhost")
+(setq emms-player-mpd-server-port "6600")
+(setq emms-player-mpd-music-directory "~/Music")
+#+end_src
+
+Add the required functions to EMMS lists:
+#+begin_src emacs-lisp
+(add-to-list 'emms-info-functions 'emms-info-mpd)
+(add-to-list 'emms-player-list 'emms-player-mpd)
+#+end_src
+
+Now we can connect EMMS to MPD. For some reason, executing this function stops the MPD playback, but it is not a big issue because it has to be executed only once.
+#+begin_src emacs-lisp
+(emms-player-mpd-connect)
+#+end_src
+
+The last thing we may want is to link clearing EMMS playlist to clearing MPD playlist. I'm not sure how this interacts with MPD's own playlists because I don't use them, so you may need to watch out here if you do.
+
+#+begin_src emacs-lisp
+(add-hook 'emms-playlist-cleared-hook 'emms-player-mpd-clear)
+#+end_src
+
+** Usage
+One rough edge of EMMS & MPD integration is that EMMS and MPD have their separate libraries and playlists.
+
+So, first we have to populate the MPD library with =M-x emms-player-mpd-update-all=. This operation is asyncronous but may take a few minutes for the first run. The subsequent runs are much faster. As an alternative, you can run =mpc update= from the command line.
+
+Second, we have to populate the EMMS library (cache) from the MPD library. To do that, run =M-x emms-cache-set-all-from-mpd=. If something went wrong with EMMS cache, you always can clean it with =M-x emms-cache-reset=.
+
+After this is done, we can finally play music! To do that, run =M-x emms-browser=. The left window should have the EMMS browser buffer with the loaded library, the right one should countain (as for now empty) playlist.
+
+In the browser we can use the following commands to add elements to the playlist:
+- =M-x emms-browser-toggle-subitems= (=<tab>= in evil, =SPC= in vanilla) to open/close the element under cursor
+- =M-x emms-browser-add-tracks= (=RET= in both styles) to add the element under the cursor to the playlist
+
+Now, we have a few tracks in the EMMS playlist, but they are not in MPD playlist yet.
+
+In the EMMS playlist buffer, =M-x emms-playlist-mode-play-smart= (=RET=) should sync the playlists and start playing the song under the cursor. Also, you can use
+- =M-x emms-playlist-mode-kill-track= (=D=) to remove the element under cursor
+- =M-x emms-playlist-clear= (=C=) to clear the playlist. With the hook from the previous section this should also clear the MPD playlist.
+
+Take a look at the [[https://www.gnu.org/software/emms/manual/][EMMS manual]] for more information, including sections about [[https://www.gnu.org/software/emms/manual/#Interactive-Playlists][playlist]] and [[https://www.gnu.org/software/emms/manual/#The-Browser][browser.]]
+
+** Fetching lyrics
+One feature of ncmpcpp I was missing here is fetching lyrics, so I've written a small package to do just that.
+
+Debugging the package turned out to be quite funny, because apparently, there is no way around HTML parsing with this task. So I've chosen genius.com as a backend, but the site turned out to provide different versions of itself (with different DOMs!) to different users.
+
+At any rate, I've processed the cases I found, and it seems to be working, at least for me. To use the package, [[https://genius.com/api-clients/new][get the api key]] from Genius and install it:
+
+#+begin_src emacs-lisp
+(use-package lyrics-fetcher
+  :straight t
+  :after (emms)
+  :config
+  (setq lyrics-fetcher-genius-access-token
+        (password-store-get "My_Online/APIs/genius.com")))
+#+end_src
+
+To fetch lyrics for the current playing EMMS song, run ~M-x lyrics-fetcher-show-lyrics~. Or run ~M-x lyrics-fetcher-emms-browser-show-at-point~ to fetch data for the current point in EMMS browser. See [[https://github.com/SqrtMinusOne/lyrics-fetcher.el][the package homepage]] for more information.
+
+** Album covers
+I've mentioned above that EMMS supports displaying album covers.
+
+For this to work, it is necessary to have one album per one folder. By default the cover image should be put in images called =cover_small= (100x100 recommended), =cover_medium= (200x200 recommended) and =cover_large=. The small version is to be displayed in the EMMS browser, the medium one in the playlist.
+
+It's not required for images be exactly of these sizes, but they definitely should be of one size across different albums to look nice in the interface.
+
+You can resize images with ImageMagick with commands like this:
+#+begin_src bash
+convert cover.jpg -resize 100x100^ -gravity Center -extent 100x100 cover_small.jpg
+convert cover.jpg -resize 200x200^ -gravity Center -extent 200x200 cover_medium.jpg
+#+end_src
+
+=lyrics-fetcher= can (try to) do this automatically by downloading the cover from genius.com with =M-x lyrics-fetcher-emms-browser-fetch-covers-at-point= in EMMS browser.
+* MPV and YouTube
+[[https://mpv.io/][MPV]] is a nice extensible media player, which can be controlled from EMMS and be used to watch YouTube videos because it integrates with [[https://github.com/ytdl-org/youtube-dl][youtube-dl]], so it fits quite nicely in this setup.
+
+** MPV and youtube-dl
+First, install both mpv in youtube-dl from your distribution's package repository.
+
+Then we can add another player to the list:
+#+begin_src emacs-lisp
+(add-to-list 'emms-player-list 'emms-player-mpv)
+#+end_src
+
+EMMS determines which player to use by a regexp. =emms-player-mpd= sets the default MPD regexp from MPD's diagnostic output so that regex opens basically everything, including videos, https links, etc. That is fine if MPD is the only player in EMMS, but as we want to use MPV as well, we need to override the regexes.
+
+MPD regexp can look like this:
+#+begin_src emacs-lisp
+(emms-player-set emms-player-mpd
+                 'regex
+                 (emms-player-simple-regexp
+                  "m3u" "ogg" "flac" "mp3" "wav" "mod" "au" "aiff"))
+#+end_src
+
+And a regexp for MPV to open videos and youtube URLs:
+#+begin_src emacs-lisp
+(emms-player-set emms-player-mpv
+                 'regex
+                 (rx (or (: "https://" (* nonl) "youtube.com" (* nonl))
+                         (+ (? (or "https://" "http://"))
+                            (* nonl)
+                            (regexp (eval (emms-player-simple-regexp
+                                           "mp4" "mov" "wmv" "webm" "flv" "avi" "mkv")))))))
+#+end_src
+
+Then, by default youtube-dl downloads the video in the best possible quality, which may be pretty high. To have some control over it, we can modify the =--ytdl-format= key in the =emms-player-mpv-parameters= variable. I've come up with the following solution:
+#+begin_src emacs-lisp
+(setq my/youtube-dl-quality-list
+      '("bestvideo[height<=720]+bestaudio/best[height<=720]"
+        "bestvideo[height<=480]+bestaudio/best[height<=480]"
+        "bestvideo[height<=1080]+bestaudio/best[height<=1080]"))
+
+(setq my/default-emms-player-mpv-parameters
+      '("--quiet" "--really-quiet" "--no-audio-display"))
+
+(defun my/set-emms-mpd-youtube-quality (quality)
+  (interactive "P")
+  (unless quality
+    (setq quality (completing-read "Quality: " my/youtube-dl-quality-list nil t)))
+  (setq emms-player-mpv-parameters
+        `(,@my/default-emms-player-mpv-parameters ,(format "--ytdl-format=%s" quality))))
+
+(my/set-emms-mpd-youtube-quality (car my/youtube-dl-quality-list))
+#+end_src
+
+Run =M-x my/set-emms-mpd-youtube-quality= to pick the required quality. Take a look at [[https://github.com/ytdl-org/youtube-dl/blob/master/README.md#format-selection][youtube-dl docs]] for more information about the format selection.
+
+Now =M-x emms-add-url= should work on YouTube URLs just fine. Just keep in mind that it will only add the URL to the playlist, not play it right away.
+** Cleanup EMMS cache
+All the added URLs stay in the EMMS cache after being played. We probably don't want them to remain there, so here is a function to remove URLs from the EMMS cache.
+
+#+begin_src emacs-lisp
+(defun my/emms-cleanup-urls ()
+  (interactive)
+  (let ((keys-to-delete '()))
+    (maphash (lambda (key value)
+               (when (eq (cdr (assoc 'type value)) 'url)
+                 (add-to-list 'keys-to-delete key)))
+             emms-cache-db)
+    (dolist (key keys-to-delete)
+      (remhash key emms-cache-db)))
+  (setq emms-cache-dirty t))
+#+end_src
+* YouTube RSS
+** Where to get URLs?
+So, we are able to watch YouTube videos by URLs, but where to get URLs from? A natual solution is to use [[https://github.com/skeeto/elfeed][elfeed]] and RSS feeds.
+
+I've tried a bunch of options to get feeds for YouTube channels. The first one is [[https://api.invidious.io/][Invidious]], a FOSS YouTube frontend. The problem here is that various instances I tried weren't particularly stable (at least when I was using them) and hosting the thing by myself would be an overkill. And switching instances is causing duplicate entries in the Elfeed DB.
+
+The second option is to use YouTube's own RSS. The feed URL looks like ~https://www.youtube.com/feeds/videos.xml?channel_id=<CHANNEL_ID>=~. [[https://stackoverflow.com/questions/14366648/how-can-i-get-a-channel-id-from-youtube][Here are]] a couple of options of figuring out =CHANNEL_ID= in case it's not easily available. The problem with YouTube RSS is that it uses fields which are not supported by elfeed, so the feed entry lacks a preview and description.
+
+As a final workaround, I've written a small [[https://github.com/SqrtMinusOne/yt-rss][web-server]] which converts an RSS from YouTube to an elfeed-compatible Atom feed. It doesn't do much, so you can just download the thing and launch it:
+
+#+begin_src bash :eval no
+git clone https://github.com/SqrtMinusOne/yt-rss.git
+cd ./yt-rss
+pip install -r requirements.txt
+gunicorn main:app
+#+end_src
+
+A feed for a particular channel will be available at
+#+begin_src
+http://localhost:8000/<channel_id>?token=<token>
+#+end_src
+
+where =<token>= is set in =.env= file to the default value of =12345=.
+** Elfeed
+[[https://github.com/skeeto/elfeed][Elfeed]] is an Emacs Atom & RSS reader. It's a pretty popular package with lots of information written over the years, so I'll cover just my particular setup.
+
+My elfeed config, sans keybindings, looks like this:
+#+begin_src emacs-lisp
+(use-package elfeed
+  :straight t
+  :commands (elfeed)
+  :config
+  (setq elfeed-db-directory "~/.elfeed")
+  (setq elfeed-enclosure-default-dir (expand-file-name "~/Downloads"))
+  (advice-add #'elfeed-insert-html
+              :around
+              (lambda (fun &rest r)
+                (let ((shr-use-fonts nil))
+                  (apply fun r)))))
+#+end_src
+
+The advice there forces elfeed to use monospace fonts in the show buffer.
+
+I also use [[https://github.com/remyhonig/elfeed-org][elfeed-org]], which gives an option to store the feed config in an =.org= file instead of a variable:
+#+begin_src emacs-lisp
+To store the feed config,
+(use-package elfeed-org
+  :straight t
+  :after (elfeed)
+  :config
+  (setq rmh-elfeed-org-files '("~/.emacs.d/elfeed.org"))
+  (elfeed-org))
+#+end_src
+
+So, however you've got URLs for YouTube channels, put them into elfeed.
+
+To fetch the feeds, open elfeed with =M-x elfeed= and run =M-x elfeed-search-fetch= in the search buffer. And as usual, take a look at [[https://github.com/skeeto/elfeed][the package documentation]] for more information.
+
+To navigate through a large list of feeds, I've made the following function to narrow the search buffer to the feed of the entry under cursor:
+#+begin_src emacs-lisp
+(defun my/elfeed-search-filter-source (entry)
+  "Filter elfeed search buffer by the feed under cursor."
+  (interactive (list (elfeed-search-selected :ignore-region)))
+  (when (elfeed-entry-p entry)
+    (elfeed-search-set-filter
+     (concat
+      "@6-months-ago "
+      "+unread "
+      "="
+      (replace-regexp-in-string
+       (rx "?" (* not-newline) eos)
+       ""
+       (elfeed-feed-url (elfeed-entry-feed entry)))))))
+#+end_src
+
+So I mostly alternate between =M-x my/elfeed-search-filter-source= and =M-x elfeed-search-clear-filter=. I tag the entries which I want to watch later with =+later=, and add the ones I want to watch right now to the playlist.
+
+** Integrating with EMMS
+Finally, but how exactly can we add an entry from elfeed to the EMMS playlist?
+
+Here's the solution I came up with. First, we've got to get a URL:
+#+begin_src emacs-lisp
+(defun my/get-youtube-url (link)
+  (let ((watch-id (cadr
+                   (assoc "watch?v"
+                          (url-parse-query-string
+                           (substring
+                            (url-filename
+                             (url-generic-parse-url link))
+                            1))))))
+    (concat "https://www.youtube.com/watch?v=" watch-id)))
+#+end_src
+
+This function is intended to work with both Invidious and YouTube RSS feeds. Of cource, it will require some adaptation if you want to watch channels from something like PeerTube or Odysee.
+
+The easiest way to put the URL to the playlist is to define a new source for EMMS:
+#+begin_src emacs-lisp
+(define-emms-source elfeed (entry)
+    (let ((track (emms-track
+                  'url (my/get-youtube-url (elfeed-entry-link entry)))))
+      (emms-track-set track 'info-title (elfeed-entry-title entry))
+      (emms-playlist-insert-track track)))
+#+end_src
+Because =define-emms-source= is an EMMS macro, the code block above has to be evaluated with EMMS loaded. E.g. you can wrap it into =(with-eval-after-load 'emms ...)= or put in the =:config= section.
+
+The macro defines a bunch of functions to work with source, which we can use in another function:
+#+begin_src emacs-lisp
+(defun my/elfeed-add-emms-youtube ()
+  (interactive)
+  (emms-add-elfeed elfeed-show-entry)
+  (elfeed-tag elfeed-show-entry 'watched)
+  (elfeed-show-refresh))
+#+end_src
+
+Now, calling =M-x my/elfeed-add-emms-youtube= in the =*elfeed-show*= buffer will add the correct URL to the playlist and tag the entry with =+watched=. I've bound the function to =gm=.