Patching elfeed and shr to handle SVG images with viewBox attributes

Posted: - Modified: | emacs

I want to use more SVG sketches in my blog posts. I would like them to look reasonable in elfeed. When I first checked them out, though, the images were a little too big to be comfortable.

To test things quickly, I figured out how to use Elfeed to display the first entry from a local file.

(with-temp-buffer (get-buffer-create "*elfeed*")
  (elfeed-show-entry
   (car
    (elfeed-entries-from-rss
     "file:///tmp/test/index.xml"
     (xml-parse-file "/tmp/test/index.xml")))))
2024-01-26_08-34-03.png
Figure 1: The image is a little too big to be comfortable

I'd just been using the default width and height from the pdftocairo import, but changing the width and height seemed like a good first step. I could fix this when I convert the file, export it from Org Mode as a my-include link, or transform it when processing it in the 11ty static site generator. Let's start by changing it in the SVG file itself.

(defun my-svg-resize (file width height)
  "Resize FILE to WIDTH and HEIGHT in pixels, keeping aspect ratio."
  (interactive "FSVG: \nnWidth: \nnHeight: ")
  (let* ((dom (xml-parse-file file))
         (orig-height (string-to-number (dom-attr dom 'height)))
         (orig-width (string-to-number (dom-attr dom 'width)))
         (aspect-ratio (/ (* 1.0 orig-width) orig-height))
         new-width new-height)
    (setq new-width width
          new-height (/ new-width aspect-ratio))
    (when (> new-height height)
      (setq new-height height
            new-width (* new-height aspect-ratio)))
    (dom-set-attribute dom 'width (format "%dpx" new-width))
    (dom-set-attribute dom 'height (format "%dpx" new-height))
    (with-temp-file file
      (xml-print dom))))

Even when I changed the width and height of the SVG image, the image didn't follow suit. Mysterious.

2024-01-26_08-34-32.png
Figure 2: SVG image cut off

After a bit of digging around using Edebug, I found out that elfeed uses shr, which uses libxml-parse-html-region, and that converts attributes to lowercase. This is generally what you want to do for HTML, since HTML tags and attributes are case-insensitive, but SVG tags are case-sensitive. It looks like other implementations work around this by special-casing attributes. libxml-parse-html-region is C code that calls a library function, so it's hard to tinker with, but I can at least fix the behaviour at the level of shr. I took the list of SVG attributes to correct case for and wrote this code to fix the attribute cases.

List of atttributes to correct
(defconst shr-correct-attribute-case
  '((attributename . attributeName)
    (attributetype . attributeType)
    (basefrequency . baseFrequency)
    (baseprofile . baseProfile)
    (calcmode . calcMode)
    (clippathunits . clipPathUnits)
    (diffuseconstant . diffuseConstant)
    (edgemode . edgeMode)
    (filterunits . filterUnits)
    (glyphref . glyphRef)
    (gradienttransform . gradientTransform)
    (gradientunits . gradientUnits)
    (kernelmatrix . kernelMatrix)
    (kernelunitlength . kernelUnitLength)
    (keypoints . keyPoints)
    (keysplines . keySplines)
    (keytimes . keyTimes)
    (lengthadjust . lengthAdjust)
    (limitingconeangle . limitingConeAngle)
    (markerheight . markerHeight)
    (markerunits . markerUnits)
    (markerwidth . markerWidth)
    (maskcontentunits . maskContentUnits)
    (maskunits . maskUnits)
    (numoctaves . numOctaves)
    (pathlength . pathLength)
    (patterncontentunits . patternContentUnits)
    (patterntransform . patternTransform)
    (patternunits . patternUnits)
    (pointsatx . pointsAtX)
    (pointsaty . pointsAtY)
    (pointsatz . pointsAtZ)
    (preservealpha . preserveAlpha)
    (preserveaspectratio . preserveAspectRatio)
    (primitiveunits . primitiveUnits)
    (refx . refX)
    (refy . refY)
    (repeatcount . repeatCount)
    (repeatdur . repeatDur)
    (requiredextensions . requiredExtensions)
    (requiredfeatures . requiredFeatures)
    (specularconstant . specularConstant)
    (specularexponent . specularExponent)
    (spreadmethod . spreadMethod)
    (startoffset . startOffset)
    (stddeviation . stdDeviation)
    (stitchtiles . stitchTiles)
    (surfacescale . surfaceScale)
    (systemlanguage . systemLanguage)
    (tablevalues . tableValues)
    (targetx . targetX)
    (targety . targetY)
    (textlength . textLength)
    (viewbox . viewBox)
    (viewtarget . viewTarget)
    (xchannelselector . xChannelSelector)
    (ychannelselector . yChannelSelector)
    (zoomandpan . zoomAndPan))
  "Attributes for correcting the case in SVG and MathML.
Based on https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-inforeign .")

This recursive function does the actual replacements.

(defun shr-correct-dom-case (dom)
  "Correct the case for SVG segments."
  (dolist (attr (dom-attributes dom))
    (when-let ((rep (assoc-default (car attr) shr-correct-attribute-case)))
      (setcar attr rep)))
  (dolist (child (dom-children dom))
    (shr-correct-dom-case child))
  dom)

Then we can replace shr-tag-svg with this:

(defun shr-tag-svg (dom)
  (when (and (image-type-available-p 'svg)
             (not shr-inhibit-images)
             (dom-attr dom 'width)
             (dom-attr dom 'height))
    (funcall shr-put-image-function
             (list (shr-dom-to-xml (shr-correct-dom-case dom) 'utf-8)
                   'image/svg+xml)
             "SVG Image")))

The result is that the image now respects width, height, and viewBox:

2024-01-26_08-33-31.png
Figure 3: Fixed image

Here is a small test for shr-correct-dom-case:

(ert-deftest shr-correct-dom-case ()
  (let ((case-fold-search nil))
    (should
     (string-match
      "viewBox"
      (shr-dom-to-xml
       (shr-correct-dom-case
        (with-temp-buffer
          (insert "<svg viewBox=\"0 0 100 100\"></svg>")
          (libxml-parse-html-region (point-min) (point-max)))))))))

And another test involving displaying an image:

(with-current-buffer (get-buffer-create "*test*")
  (erase-buffer)
  (insert "<svg width=\"100px\" height=\"100px\" viewBox=\"0 0 200 200\"><circle cx=\"100\" cy=\"100\" r=\"100\"/></svg>\n")
  (shr-insert-document (libxml-parse-html-region (point-min) (point-max)))
  (display-buffer (current-buffer)))
2024-01-26_09-09-48.png
Figure 4: Correct output: full circle

shr.el is in Emacs core, so I'll need to turn this into a patch and send it to emacs-devel at some point.

View org source for this post

My Evil Plan for Yay Emacs!

| yay-emacs, emacs, planning

Here's a clip from my 2024-01-21 Yay Emacs livestream about my goals for Yay Emacs and the built-in payoffs that I think will help me keep doing it.

  • 00:00.000: Getting more ideas into blog posts and workflow demos: Now, also, I have an evil plan. My evil plan is that this is a good way for me to get ideas and convert them into blog posts and code and then do the workflow demos, because it's sometimes really difficult to see how to use something from just the code.
  • 00:20.340: I have fun tickling my brain: This process is fun. Tweaking Emacs is fun for me.
  • 00:24.840: I learn from other people's comments, questions: Also, if I do this out loud, other people can help out with questions and comments, like the way that you're all doing now, which is great. Fantastic.
  • 00:35.200: Other people pick up ideas: Of course, those are all very selfish reasons. So I'm hoping other people are getting something out of this too. (Hello, 19 people who are watching, and also for some reason, the hundreds of people who check these videos out afterwards. Great, fantastic.) I'm hoping you pick up some ideas from the crazy things that we like to play with in terms of Emacs.
  • 00:57.440: We bounce ideas around and make lots of progress: My medium-term plan there is then to start seeing how those ideas get transformed when they get bounced off other people and other people bounce ideas back. Because that's one of the fun things about Emacs, right? Everything is so personalizable that seeing how one workflow idea gets transformed into somebody else's life, you learn something from that process. I'm really looking forward to how bouncing ideas around will work here.
  • 01:32.200: More people share more: Especially if we can find little things that make doing things more fun or they make it take less effort–then maybe more people will share more things, and then I get to learn from that also,
  • 01:46.940: Building up an archive: which is fantastic because long term, this can help build up an archive. Then people can go into that archive and find things without necessarily waiting for me. I don't become the bottleneck. People can just go in there and find… "Oh, you remember that time that I saw this interesting idea about SVG highlighting or whatever." You can just go in there and try to find more information.
  • 02:11.640: More people join and thrive in the Emacs community: So that's great. Then ideally, as people find the things that resonate with them, the cool demos that say this text editor can be extended to do audio editing and animation and all that crazy stuff, then more people will come and join and share what they're learning, and then move on to building stuff maybe for themselves and for other people, and then it'll be even more amazing.
  • 02:42.780: I could be a voice in people's heads: And lastly, this is kind of odd, but having listened in the background to so many of the kiddo's current viewing habits, her favorite YouTube channels like J Perm or Cubehead or Tingman for Rubik's cube videos or Eyecraftmc for Minecraft, I'm beginning to appreciate kind of the value of having these mental models of other people in your head. I can imagine how they talk and all that stuff. I am looking forward to watching more Emacs videos, which I haven't done in a while because usually for Emacs News, I'm just skimming through the transcripts super quickly on account of (A), lots of videos and (B), not much time. So this idea of getting other people's voices into your head, or possibly becoming a voice in somebody's head, I think there might be something interesting there. Of course, the buzz these days is, "oh no, AI voice cloning, this is a safety issue and all of that stuff." But I think there are positive uses for this as well, in the sense that… As qzump says, you know, they are like, "I have no Emacs friends and you're speaking to my soul." A lot of us are doing this in isolation. We don't normally meet other people. So the more voices we can have in our heads of actual people who enjoy doing these things, the less weird we feel. Or actually, more like… the more weird we feel, but in a good way, like there's a tribe, right? If sharing more ideas in a multimedia sort of way, like with either audio narration with images or this webcam thing that we're trying (my goodness, I have to actually dress up) helps people build these mental models in their head… Hey, one of the nice things about this webcam thing is I can make hand gestures. Cool, cool. Might be interesting. If, while you're hacking on Emacs, you can imagine me cheering you on and saying, "That's fantastic. Have you thought about writing a blog post about that so that we get that into Planet Emacs Life and, and then into Emacs News? Please share what you're learning." It'll be great. So, yeah, maybe that's a thing. So that's my evil plan for Yay Emacs.
View org source for this post

2024-01-22 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Hacker News, lobste.rs, kbin, programming.dev, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post

Yay Emacs: Using elisp: links in Org Mode to note the time and display messages on stream

| yay-emacs, org

I like adding chapters to my videos so that people can jump to sections. I can figure out the sections by reading the transcript, adding NOTE comments, and extracting the times for those with my-youtube-copy-chapters. It could be nice to capture the times on the fly. org-timer could let me insert relative timestamps, but I think it might need some tweaking to synchronize that with when the stream starts according to YouTube. I've set up a capture, too, so I can take notes with timestamps.

It turns out that I don't have a lot of mental bandwidth when I'm on stream, so it's hard to remember keyboard shortcuts. (Maybe if I practise using the hydra I set up…) Fortunately, Org Mode's elisp: link type makes it easy to set up executable shortcuts. For example, I can add links like [[elisp:my-stream-message-link][TODO]] to my livestream plans like this:

2024-01-20-elisp-links.svg
Figure 1: Shortcuts with elisp: links

I can then click on the links or use C-c C-o (org-open-link-at-point) to run the function. When I follow the TODO link in the first item, Emacs displays a clock and a message based on the rest of the line after the link.

2024-01-20-message.svg
Figure 2: Displaying a clock and a message

In the background, the code also sets the description of the link to the wall-clock time.

2024-01-20-time.svg
Figure 3: Link description updated with the time

If I start the livestream with a clock displayed on screen, I can use that to translate wall-clock times to relative time offsets. I'll probably figure out some Elisp to translate the times automatically at some point, maybe based on something like org-timer-change-times-in-region.

I figured it might be fun to add a QR code automatically if we detect a URL, taking advantage of that qrencode package I started playing around with.

2024-01-20-qr.svg
Figure 4: With a QR code

You can also use elisp: links for more complicated Emacs Lisp functions, like this: elisp:(progn ... ...).

Here's the code that makes it happen. It's based on emacsconf-stream.el.

(defvar my-stream-message-buffer "*Yay Emacs*")
(defvar my-stream-message-timer nil)

(defun my-stream-message-link ()
  (interactive)
  (save-excursion
    (when (and (derived-mode-p 'org-mode)
               (eq (org-element-type (org-element-context)) 'link))
      (my-stream-update-todo-description-with-time)
      (goto-char (org-element-end (org-element-context)))
      (my-stream-message (org-export-string-as (buffer-substring (point) (line-end-position)) 'ascii t)))))
(defun my-stream-update-todo-description-with-time ()
  (when (and (derived-mode-p 'org-mode)
             (eq (org-element-type (org-element-context)) 'link))
    (my-org-update-link-description (format-time-string "%-I:%M:%S %p"))))

(defun my-stream-message (&optional message)
  (interactive "MMessage: ")
  ;; update the description of the link at point to be the current time, if any
  (switch-to-buffer (get-buffer-create my-stream-message-buffer))
  (erase-buffer)
  (delete-other-windows)
  (when (string= message "") (setq message nil))
  (face-remap-add-relative 'default :height 200)
  (insert
   "Yay Emacs! - Sacha Chua (sacha@sachachua.com)\n"
   (propertize
    "date"
    'stream-time (lambda () (format-time-string "%Y-%m-%d %H:%M:%S %Z (%z)")))
   "\n\n"
   message)
  ;; has a URL? Let's QR encode it!
  (when-let ((url (save-excursion
                    (when (re-search-backward ffap-url-regexp nil t)
                      (thing-at-point-url-at-point)))))
    (insert (propertize (qrencode url) 'face '(:height 50)) "\n"))
  (insert  "\nYayEmacs.com\n")
  (when (timerp my-stream-message-timer) (cancel-timer my-stream-message-timer))
  (my-stream-update-time)
  (setq my-stream-message-timer (run-at-time t 1 #'my-stream-update-time))
  (goto-char (point-min)))

(defun my-stream-update-time ()
  "Update the displayed time."
  (if (get-buffer my-stream-message-buffer)
      (when (get-buffer-window my-stream-message-buffer)
        (with-current-buffer my-stream-message-buffer
          (save-excursion
            (goto-char (point-min))
            (let (match)
              (while (setq match (text-property-search-forward 'stream-time))
                (goto-char (prop-match-beginning match))
                (add-text-properties
                 (prop-match-beginning match)
                 (prop-match-end match)
                 (list 'display
                       (funcall (get-text-property
                                 (prop-match-beginning match)
                                 'stream-time))))
                (goto-char (prop-match-end match)))))))
    (when (timerp my-stream-message-timer)
      (cancel-timer my-stream-message-timer))))

Let's see if that makes it easy enough for me to remember to actually do it!

View org source for this post

Running the current Org Mode Babel Javascript block from Emacs using Spookfox

| emacs, org, spookfox

I often want to send Javascript from Emacs to the web browser. It's handy for testing code snippets or working with data on pages that require Javascript or authentication. I could start Google Chrome or Mozilla Firefox with their remote debugging protocols, copy the websocket URLs, and talk to the browser through something like Puppeteer, but it's so much easier to use the Spookfox extension for Mozilla to execute code in the active tab. spookfox-js-injection-eval-in-active-tab lets you evaluate Javascript and get the results back in Emacs Lisp.

I wanted to be able to execute code even more easily. This code lets me add a :spookfox t parameter to Org Babel Javascript blocks so that I can run the block in my Firefox active tab. For example, if I have (spookfox-init) set up, Spookfox connected, and https://planet.emacslife.com in my active tab, I can use it with the following code:

#+begin_src js :eval never-export :spookfox t :exports results
[...document.querySelectorAll('.post > h2')].slice(0,5).map((o) => '- ' + o.textContent.trim().replace(/[ \n]+/g, ' ') + '\n').join('')
#+end_src
  • Mario Jason Braganza: Updated to Emacs 29.2
  • Irreal: Zamansky: Learning Elisp #16
  • Tim Heaney: Lisp syntax
  • Erik L. Arneson: Many Posts of Interest for January 2024
  • William Denton: Basic citations in Org (Part 4)

Evaluating a Javascript block with :spookfox t

To do this, we wrap some advice around the org-babel-execute:js function that's called by org-babel-execute-src-block.

(defun my-org-babel-execute:js-spookfox (old-fn body params)
  "Maybe execute Spookfox."
  (if (assq :spookfox params)
      (spookfox-js-injection-eval-in-active-tab
       body t)
    (funcall old-fn body params)))
(with-eval-after-load 'ob-js
  (advice-add 'org-babel-execute:js :around #'my-org-babel-execute:js-spookfox))

I can also run the block in Spookfox without adding the parameter if I make an interactive function:

(defun my-spookfox-eval-org-block ()
  (interactive)
  (let ((block (org-element-context)))
    (when (and (eq (org-element-type block) 'src-block)
               (string= (org-element-property :language block) "js"))
      (spookfox-js-injection-eval-in-active-tab
       (nth 2 (org-src--contents-area block))
       t))))

I can add that as an Embark context action:

(with-eval-after-load 'embark-org
  (define-key embark-org-src-block-map "f" #'my-spookfox-eval-org-block))

In Javascript buffers, I want the ability to send the current line, region, or buffer too, just like nodejs-repl does.

(defun my-spookfox-send-region (start end)
  (interactive "r")
  (spookfox-js-injection-eval-in-active-tab (buffer-substring start end) t))

(defun my-spookfox-send-buffer ()
  (interactive)
  (my-spookfox-send-region (point-min) (point-max)))

(defun my-spookfox-send-line ()
  (interactive)
  (my-spookfox-send-region (line-beginning-position) (line-end-position)))

(defun my-spookfox-send-last-expression ()
  (interactive)
  (my-spookfox-send-region (save-excursion (nodejs-repl--beginning-of-expression)) (point)))

(defvar-keymap my-js-spookfox-minor-mode-map
  :doc "Send parts of the buffer to Spookfox."
  "C-x C-e" 'my-spookfox-send-last-expression
  "C-c C-j" 'my-spookfox-send-line
  "C-c C-r" 'my-spookfox-send-region
  "C-c C-c" 'my-spookfox-send-buffer)

(define-minor-mode my-js-spookfox-minor-mode "Send code to Spookfox.")

I usually edit Javascript files with js2-mode, so I can use my-js-spookfox-minor-mode in addition to that.

I can turn the minor mode on automatically for :spookfox t source blocks. There's no org-babel-edit-prep:js yet, I think, so we need to define it instead of advising it.

(defun org-babel-edit-prep:js (info)
  (when (assq :spookfox (nth 2 info))
    (my-js-spookfox-minor-mode 1)))

Let's try it out by sending the last line repeatedly:

Sending the current line

I used to do this kind of interaction with Skewer, which also has some extra stuff for evaluating CSS and HTML. Skewer hasn't been updated in a while, but maybe I should also check that out again to see if I can get it working.

Anyway, now it's just a little bit easier to tinker with Javascript!

View org source for this post
This is part of my Emacs configuration.

Large language models and me

| ai, geek

Summary (1895 words): There's a lot of hype around large language models (LLMs) or generative pre-trained transformers (GPTs) like ChatGPT. I'm still trying to figure out how I can use them.

Text from sketch

I'm a little curious about how large language models like ChatGPT might fit into my life. There's a lot of hype, and I'm not yet sure what the value might be for me. While companies rush around trying to stake their claims and strike gold, I can find my own way more slowly.

Things it might be good at:

  • text classification with vectors
  • tip of the tongue: remembering common words and concepts
  • research and summaries
  • outline braindumps
  • follow-up questions
  • figuring out what I want (mostly in contrast to what I don't want)
  • ideas about common topics
  • making up stories and poems for the kiddo

Probably medium-term, after livestreaming and vector search.

Maybe focus on API access versus ChatGPT Plus.

It's a bit of a gold rush in the tech industry at the moment with all sorts of companies scrambling to figure out generative artificial intelligence and large language models. I feel like I don't quite line up with the personas these companies are building for. I don't want autogenerated marketing fluff for my newsletter or social media posts, and I'm still a little iffy on code. I probably need to go figure things out for myself.

I've built a couple of question-answering prototypes using retrieval-augmented generation with LangChain or Llama Index for my consulting clients, and I've started experimenting with using HuggingFace embeddings to do vector search across my blog post and sketchnote titles. I'm also curious about how other Emacs geeks have been experimenting with large language models, such as Abhinav Tushar's EmacsConf 2023 talk on simplifying data visualization with Org Babel and Matplot, Andrew Hyatt's llm package for standardizing interfaces, and the AI category items in my Emacs News.

W- has been experimenting with ChatGPT Plus too. He finds that it's useful for summarizing videos and world news. The sequence of images it generated for a Vim GPT logo was hilariously off-base. Maybe I should mentally file it under entertainment instead of productivity, and anything actually useful can be a bonus.

I think that at the moment, GPTs generally do better when you keep them. Here are some quick notes:

Text classification with vectors

I categorize Emacs News links so that it's easy for people to find the topics they're interested in. Most of the time, a simple regular expression match is enough (ex: Org Mode, Denote, Magit). I can probably make a training data set from recent Emacs News posts so that I can use text classification libraries to suggest a category for links that aren't easily figure out with regular expressions.

Tip of the tongue

I think GPT is useful for figuring out the words to look for. One of the challenges of learning more is figuring out the jargon, like learning how to use M-x apropos to look for things related to buffers. (Apropos, buffers; Emacs uses lots of weird words.) Even when you know the words, it can be hard to remember them: tip of the tongue is a thing, and it'll probably get harder as I get older. (Another good reason to exercise.) It'll be handy if I can just vaguely describe something and then get the words to look for, or use vector search to find things based on similarity rather than exact matches.

Research and summarization

I appreciate that the web access in ChatGPT-4 and Microsoft Bing/Copilot allows me to get search results back without having to deal with all the generic SEO content, newsletter popups, and content paywalls that plague the modern web. Sometimes it will even analyze the information and put things into tables or outlines for me. Of course, it is also contributing to the problem by making it easier for people to churn out lots of generic SEO content or newsletter updates, so this feels somewhat self-defeating in the long run. We'll just have to see how it all works out.

I'm also curious about using it to summarize the videos I come across for Emacs News, although people usually set good titles and a quick skim of the transcript usually helps.

Outlines

I'm ambivalent about generating outlines for blog post ideas. The ones it comes up with from short prompts are pretty generic and intro-focused, and it feels easier to just start with my own outline.

Sample outline for a blog post idea

I. Introduction

  1. Personal Reflections
    • Express curiosity about the potential of large language models in daily life.
    • Discuss the hype around these models versus personal exploration.
  2. Setting the Stage
    • Briefly mention the rise of companies in this domain and public interest.

II. Understanding Large Language Models

  1. What are Large Language Models?
    • Definition and basic explanation.
    • Brief history and evolution.
  2. ChatGPT: A Case Study
    • Introduction to ChatGPT.
    • How it differs from traditional models.

III. Practical Applications of ChatGPT

  1. Everyday Uses
    • Text classification with vectors.
    • Memory assistance: Remembering common words and concepts.
    • Research and summaries.
  2. Creative Endeavors
    • Generating stories and poems.
    • Using ChatGPT for brainstorming and idea generation.
  3. Professional Scenarios
    • Outlining and organizing thoughts.
    • Follow-up questions and deep-dives into topics.
    • Contrasting desires and needs: Figuring out preferences.

IV. Exploring Advanced Features and Accessibility

  1. Beyond Basic Use: Advanced Features
    • Exploring vector search.
    • Potential of ChatGPT in livestreaming contexts.
  2. Accessibility and Options for Users
    • Comparing API access with ChatGPT Plus.
    • Discussing the implications for different types of users.

V. Conclusion

  1. Personal Reflections Revisited
    • Revisiting initial skepticism or curiosity.
    • Reflections based on the exploration in the article.
  2. Looking Forward
    • Potential future developments in large language models.
    • Encouraging reader interaction: Inviting questions and comments.

The outlines it generates from my audio braindumps could be an interesting way to structure a large wall of text, if I could just get it to use more of my own words while still correcting misrecognized words.

Follow-up questions and ideas

I could give GPT a blog post or presentation draft and ask it to generate follow-up questions so that I can flesh ideas out further. Although usually adding more stuff isn't a problem for me–it's more like finding a good stopping point instead of getting tempted to squeeze one more hack in…

The kiddo sometimes needs a bit of prompting to elaborate on her writing homework. She responds well to me mind-mapping while we free-associate words, but sometimes it helps to have follow-up questions, and it works out much better when the computer's the one asking the follow-up questions instead of me.

Figuring out what I want to say and how

Sometimes I'm not happy with the way a sentence or paragraph flows. Metaphors can be helpful. Mixed metaphors are weird. Maybe a GPT can help me find a smoother way to explain things. Getting writing suggestions seems to be fine on a very small scale (phrases or sentences).

When I ask GPT to write longer things, then things get crazy. ChatGPT is pretty good at getting me to say "That totally doesn't sound like me, I'd better just write it myself," which seems like a common result.

AI style guide

I wanted to see if I could get it to sound more like me, so I went through Tiago Forte's tutorial for creating an AI style guide. I gave it a few of my recent blog posts (1, 2, 3, and 4) and asked it to summarize the resulting style guide concisely. I've included the details in case you want to get an idea of the kinds of things it makes, but it seems more suited to flattery than to practical application.

Sample analysis for one blog post

Voice and Tone

  • Formality: The language is informal and conversational, combining personal anecdotes with a discussion of workflows and projects.
  • Authoritative vs. Friendly: The tone is friendly and reflective, sharing personal experiences and thoughts in an approachable manner.
  • Overall Tone: There is a sense of openness and exploration, with a focus on sharing personal learning and development processes.

Mood

  • The mood is contemplative and optimistic, reflecting a mindset of constant learning and adaptation.

Sentence Structure

  • Structure Type: A mix of compound and complex sentences, with a strong narrative flow that interweaves personal insights with technical descriptions.
  • Length: The sentences vary in length, balancing brief statements with more elaborate explanations.

Transition Style

  • Transitions are smooth and natural, often moving from a general idea to specific examples or from one aspect of a project to another.
  • The writer uses a blend of direct transitions and narrative progression to guide the reader through different ideas and concepts.

Rhythm and Pacing

  • The rhythm is consistent and engaging, with a good balance between detailed technical information and personal narrative.
  • The pacing is moderate, allowing the reader to absorb the information without feeling rushed.

Signature Styles

  • Recurring Phrases: Frequent use of terms related to personal productivity, technology tools, and creative processes.
  • Unique Punctuation: Inclusion of lists and occasional asides to provide additional context or to elaborate on a point.
  • Patterns/Repetitions: Regular mention of personal routines and habits, combined with technical tool usage and idea development.

Distinctive Writing Style Indicators

  • A strong focus on personal productivity and idea management, combined with technical tool usage.
  • Frequent reflection on personal learning processes and adaptation strategies.
  • An engaging narrative style that seamlessly combines personal experiences with technical advice and insights.
  • The inclusion of potential future improvements and explorations, indicating a forward-thinking and open-ended approach to problem-solving.

In summary, this text further cements the author's distinctive style of blending personal narrative with technical content. The writing is reflective, engaging, and informative, offering insights into personal workflows and idea management. The author's approachable tone and focus on continuous learning and adaptation make complex topics accessible and relatable to the reader.

Expanded AI style guide combining the guides from several posts

Voice and Tone

  • Formality: Adopt a semi-formal language, leaning towards informal. Use a conversational style but include technical jargon where appropriate.
  • Authoritative vs. Friendly: Aim for a friendly, informative tone. The writing should guide and assist, rather than command.
  • Overall Tone: Maintain a helpful and instructive tone, with a focus on practical application and reader engagement.

Mood

  • Keep the mood pragmatic and enthusiastic, especially when discussing the efficiency and utility of tools, methods, and personal projects.

Sentence Structure

  • Structure Type: Utilize a mix of simple, compound, and complex sentences.
  • Length: Vary sentence lengths. Use brief sentences for straightforward facts and longer ones for detailed technical explanations and personal anecdotes.

Transition Style

  • Ensure transitions are smooth, moving from general ideas to specific examples. Use both direct transitions and subtle shifts in topic, based on the subject matter.

Rhythm and Pacing

  • Maintain a moderately paced rhythm. Balance shorter sentences for clear facts and longer ones for in-depth explanations.
  • Keep the narrative conversational and engaging, with a balance between technical details and personal reflections.

Signature Styles

  • Recurring Phrases: Frequently use phrases like "Org Mode," "Emacs," and relevant technical terms.
  • Unique Punctuation: Use parentheses for additional information and include code snippets where relevant.
  • Patterns/Repetitions: Regularly follow the introduction of a concept with practical, often technical, examples.

Distinctive Writing Style Indicators

  • Blend personal experience with technical instruction.
  • Use specific examples, particularly code snippets, to illustrate points.
  • Combine technical jargon with a conversational tone.
  • Include practical applications and benefits of described methods or tools.
  • Integrate personal anecdotes and reflections with technical content.
  • Use interactive elements like reader questions and responses to foster a community-focused approach.

Additional Elements

  • Focus on problem-solving and exploring new tools or methods.
  • Include visual elements like screencasts or diagrams to enhance understanding.
  • Reflect on personal learning processes and adaptation strategies.
  • Consider potential future improvements and explorations in the subject matter.

When I asked it to summarize the style in a single paragraph, it said:

Use semi-formal, conversational language with occasional technical jargon. The tone should be friendly and informative, focusing on practical applications and engaging the reader. Include a mix of sentence structures, varying lengths for clarity and detail. Transition smoothly between general ideas and specific examples, maintaining a moderate pace with a balance of technical details and personal reflections. Frequently use phrases like 'Org Mode' and 'Emacs', and incorporate parentheses for additional information and code snippets to illustrate points. Blend personal experiences with technical instruction, using specific examples, especially code, to clarify concepts. The content should emphasize problem-solving, exploring new tools/methods, and personal learning processes, with visual elements like diagrams for better understanding. Aim to reflect on potential future improvements in the subject matter, fostering a community-focused approach with interactive elements like reader questions.

This feels related to the Barnum effect, which is the psychological phenomenon that explains how many people feel like very general statements (ex: horoscopes) are specific to them. I'm not sure I could give this to a human and get them to write like me enough that I'd want to use the text.

I've had a few opportunities to reflect on what does sound like me, mostly thanks to EmacsConf. For example, I definitely skew the e-mail personality away from polished to personal. (Do other conference organizers include "aaaah!" and stress levels in their e-mails?) I tell GPT to get rid of its adjectives. Even with this style guide and additional instructions like avoiding adjectives, I haven't gotten ChatGPT to cut down on this thing I can't quite figure out how to explain. The way it writes feels like it's just a bit much.

Generating text

Here's what it came up with when I asked it to use the style guide and write one paragraph about how the cat's been waking me up at 5:30 AM, the kiddo wants bedtime snuggles at 11:30 PM, and I can tell when the sleep deprivation's building up thanks to the brain hiccups I get when solving a Rubik's cube. It said:

Ah, the joys (and challenges) of balancing family life with personal interests and hobbies! Let me tell you, as someone who's navigated the tricky waters of being woken up by a cat at the ungodly hour of 5:30 AM and then, barely blinking away the sleep, being summoned for late-night snuggle sessions with the kiddo until 11:30 PM, I totally get the feeling of being perpetually sleep-deprived. This exhaustion hits hardest when I'm trying to solve Rubik's cubes. Normally, I'd zip through the algorithms with ease, but now, I find my brain hiccupping mid-solve, forcing me to backtrack and re-solve. It's like my mind is stuck in a loop!

Which… well… It's reassuring to know that even though I sometimes feel self-conscious about enthusiasm! exclamation marks! geeking out about stuff! I actually write in a more restrained manner compared to how ChatGPT thinks I write.

Also, ChatGPT enthusiastically went on to write two paragraphs making up more stuff out of whole cloth, which I will tuck behind a collapsible section because this is definitely more along the lines of entertainment rather than productivity at this time.

More from ChatGPT

But, you know, this is where my geek side comes to the rescue. I've started using Org Mode in Emacs (my go-to productivity tool) to manage my day better. How? Well, for starters, I've programmed a custom routine that reminds me to take short, but effective power naps. Think (setq power-nap '(15 "minutes")). This tiny snippet of code, embedded in my daily schedule, works wonders! Plus, Org Mode's flexibility allows me to seamlessly integrate family time into my daily planner, ensuring I don't miss out on those precious moments. And when it comes to problem-solving, I've begun to treat my Rubik's cube solves like mini projects. I break them down into steps and track my progress (and missteps) in Emacs. This not only helps me understand where my sleep-deprived brain is tripping up but also serves as a great way to visualize my thought process (hello, mind maps!).

Looking ahead, I'm considering exploring Emacs extensions that could further optimize my routine, maybe even one that predicts the best times for cube-solving based on my sleep patterns. It's all about finding that sweet spot where personal hobbies and family life can coexist without leaving me feeling like a zombie. Who knows, maybe I'll even code a mini-game in Emacs for my kiddo, blending playtime with my love for coding. The possibilities are endless, and that's the beauty of it. Any fellow Emacs users out there with similar challenges? How do you balance it all? Let's share some tips and tricks!

So yeah. Maybe not text generation just yet.

Ideas about common topics

My energy can get pretty low during the afternoon because of the aforementioned 5:30 cat alarm. It can help to have something suggest a list of ideas that we can pick from, like little Minecraft build challenges.

Making up stories and poems for the kiddo

It's a handy way to generate short stories and poems based on the kiddo's interests. I've been getting her to supply a prompt in either the SWBST story summary format ("Somebody… Wanted… But… So… Then") or the beginning/middle/end format so she does some of the work, and then GPT can deliver the payoff in the form of a fleshed-out story that invariably makes us giggle.

Wrapping up

This is more of a low-priority medium-term thing for me to explore, probably after I spend some more time getting the hang of livestreaming, transcripts, and vector search. I'll keep an eye out for interesting ideas.

I think I can access the GPT 4 model (or maybe gpt-4-1106-preview) on a pay-per-use basis through the API, which will probably work out better than paying USD 20/month for ChatGPT Plus. I can also check out Gemini Pro's API.

A cursory skim of r/LocalLLaMA gives me the impression that for reasonable non-toy use, local large language models require way better hardware than I have at the moment. Even people with more powerful hardware tend to focus on smaller models (maybe for coding autosuggestions or generating text for roleplaying games), and they use commercial GPTs for things where they want larger context windows or more general knowledge. Might be worth checking on this in a little while as people work on scaling models down and figuring out more things to do with open source models.

Anyway, those are my notes on this so far. Looking forward to reading more about how people are using LLMs in Emacs, considering all the things that Emacs can work with!

View org source for this post

Emacs tweaks: Choosing what to hack on

Posted: - Modified: | emacs, life, productivity

[2024-01-22 Mon]: How do you combat being overwhelmed by choice? is somewhat relevant. I particularly like this comment which talks about delaying the decision to see if it still makes sense.

Is it the Emacs lifecycle that you tweak your config for few months and then you live off of fat of the land for >4 years? My Emacs config is a Org-tangle spaghetti that I touch only if I want to set some more sane config variable.

@xgqt@emacs.ch

This got me thinking about how tweaking my config fits in with other things I want to do–how I choose what to hack on and for how long.

Text from sketch

Choosing what to hack on - 2024-01-16-01

When it comes to computer work, I can usually choose what to do.

Idea -> Test

XKCD on automation: time to automate vs frequency x time saved

I focus more on what I'll enjoy (both the destination and the journey)

  1. How can I make this better?
  2. What's the smallest step I can take? What can I fit in 15-30 minutes?
  3. What's nearby?
    • Relevant functions or packages
    • Next steps, possibilities
  4. What kinds of notes can I leave for myself or others?
  5. like desire paths: where, what size
  6. or like a river wearing down rocks

Sometimes when I chat with other people about automation, this XKCD chart about Is It Worth the Time? comes up.

I realized that this isn't quite how I consider things. I'm lucky in that when it comes to computer things, I get to choose most of the things I spend my time on. My consulting clients have very long wishlists that I pick from based on interests and priority, and I play with Emacs for fun.

Drawing of a task sandwich with Emacs tweaking, the actual task, and blog posts and comments

Because I enjoy tinkering around with Emacs, I often build a little Emacs hacking into my tasks. 15 or 20 minutes of exploring an idea can make it even more fun to do the actual task it's supposed to help with because then I want to test it out. Then after the task is done, I get to write about it. It's like making a little task sandwich with really nice bread. This is also a little related to sharpening the saw, which is pretty fun in Emacs. (Vim people do it too!)

These little changes add up over time, making things even more enjoyable. It's a little like the way desire paths show where people actually walk between buildings and give a sense of how much they are used, or how rivers smooth down the edges of stones. The easier I make something, the more likely I am to do it, and the more I'll get to enjoy the results of my code. It's a little like the Igors described in this essay.

When I think about something I might tweak about my Emacs configuration, I usually consider the following:

1. How can I make this better?

I like looking for ways to reduce manual work or looking-up. I tend to have a hard time with tedious, repetitive tasks. I also keep an eye out for things I've been meaning to learn.

2. What's the smallest step I can take? What can I fit in 15-30 minutes?

Small steps make it easy to squeeze in things here and there. I know my brain's going to suggest half a dozen things along the way, so it helps to start as small as possible and capture most of the other things in my inbox for later. That way, I can get to experience the benefits right away without feeling lost.

Another advantage of picking really small tasks and using Org Mode to capture the rest of the ideas is that I can try to avoid the Ovsiankina effect.1 I spend most of my day taking care of our 7-year-old, so I squeeze in my focused-time tasks early in the morning before she wakes up. Sometimes I have little opportunities to work on things throughout the day, like when she wants to read a book or watch a video. She might do that for 15-30 minutes before wanting to connect again. If I pick the wrong-sized task or I don't dump enough rough notes into my inbox so that I can get the open loops out of my head and trust that I can pick things up again, the unfinished part pulls on my brain and makes it harder to enjoy time with her. Then I get tempted to let her binge-watch Minecraft or Rubik's cube videos2 so that I can finish a thought, which doesn't quite feel like good parenting.

Lastly, I don't usually understand enough about my needs to build something complex from the start. Trying things out helps me discover more about what's possible and what I want.

3. What's nearby?

Thanks to Emacs's amazing community, there are usually relevant functions or packages that I can borrow code from. I mostly have a sense of things from the blog posts and forum threads that cross my radar because of Emacs News, and I should probably get used to skimming the descriptions in the "New packages" list because that can help me find even more things.

When coming up with possible approaches, I also sometimes think about other related ideas I've had before. Filing those ideas into the appropriate subtrees in my Org files sometimes helps me come across them again. If I can take a small step that also gets me closer to one of those ideas, that's handy.

I also like to think about next steps and possibilities. For example, even if I spend an hour or two learning more about data visualization with Org Mode and plotting, that's something I can use for other things someday. This works pretty well with keeping things small, too, since small parts can be combined in surprisingly interesting ways.

Let me try to trace through a web of related features so I can give you a sense of how this all works in teeny tiny steps.

G defun defun my-include:...?from-regexp=...&to-regexp... my-include:...?from-regexp=...&to-regexp... defun->my-include:...?from-regexp=...&to-regexp... my details my details defun->my details defvar defvar defun->defvar emacsconf-el emacsconf-el defun->emacsconf-el context context defun->context my-include:...?name= my-include:...?name= my-include:...?from-regexp=...&to-regexp...->my-include:...?name= :summary :summary my details->:summary defun-open defun-open :summary->defun-open web links web links emacsconf-el->web links Embark Embark :comments both :comments both Embark->:comments both QR code QR code Embark->QR code :comments both->context web links->Embark

  • defun: I often wanted to write about a specific function, so I wrote some code to find the function definition and copy it into my export post hidden inside a details tag with the first line of the docstring as the summary. 2023-01-02
  • my-include:...?from-regexp=...&to-regexp...: Sometimes I wanted to write about longer pieces of code. I wanted to include code without repeating myself. The regular #+INCLUDE can handle line numbers or headings, but neither of them worked for the Elisp files I referred to since the line numbers kept changing as I edited the code above it and it wasn't an Org Mode file. I made my own custom link so I could specify a start and end regexp. 2023-01-08
  • my_details: I wanted to put the code in a details element so that it could be collapsible. I made an org-special-blocks template for it. special-blocks
  • :summary: For Org source blocks, I wanted to be able to do that kind of collapsible block by just adding a :summary attribute. 2023-01-27
  • defun-open: I wanted to sometimes be able to keep the function definition expanded. 2023-09-12
  • emacsconf-el: Since I was writing about a lot of EmacsConf functions in preparation for my presentation, I wanted a quick way to link to the files in the web-based repository. 2023-09-12
  • defvar: Made sense to include variable definitions too.
  • web links: The emacsconf-el links were so useful, I wanted to be able to use that type of link for other projects as well. 2024-01-07
  • Embark: I wanted to be able to copy the final URL from a custom link at point, so I used Embark. 2024-01
  • QR code: I started livestreaming again, so I wanted a quick way for viewers to get the URL of something without waiting for stream notes. 2024-01-10
  • :comments both: While scanning Reddit to find links for Emacs News, I learned about :comments both and how that includes references to the Babel file that tangled the code. 2024-01-07
  • context: Now that it was easy to link to the web version of an Emacs Lisp file, I thought it might be fun to be able to automatically include a context link by passing link=1. I also wanted to be able to navigate to the Org source code for a tangled function. 2024-01-11
  • my-include:...?name=...: I wanted to be able to refer to Org Babel source blocks by name.
I promise it's all connected. (Pepe Silva meme)

In the course of writing this blog post, I learned how to use URLs in Graphviz, learned how to include inline HTML for export with @@html:...@@, used position: sticky, figured out how to highlight the SVG using JS, used CSS to make a note that should only show up in RSS feeds, and submitted a pull request for meme.el that was merged. And now I want to figure out sidenotes or at least footnotes that don't assume they're the only footnotes on the page… This is just how my brain likes to do things. (Oooh, shiny!)

4. What kinds of notes can I leave for myself or others?

I might take years before revisiting the same topic, so good notes can pay off a lot. Also, when I share what I've been working on, sometimes people e-mail me or comment suggesting other things that are nearby, which is a lot of fun. The ideas I come up with are probably too weird to exactly line up with other people's interests, but who knows, maybe they're close enough to what other people work on that they can save people time or spark more ideas.

Inspired by Mats Lidell's EmacsConf 2023 talk on writing test cases, I've been working on writing occasional tests, too, especially when I'm writing a small, function to calculate or format something. That's a good way of sketching out how I want a function to behave so that I can see examples of it when I revisit the code. Tests also mean that if I change things, I don't have to worry too much about breaking important behaviours.

Ideas for next steps

How can I get even better at this?

  • Popping the stack (untangling interruptions and ideas): When I let myself get distracted by a cool sub-idea, I sometimes have a hard time backing up. I can get back into the habit of clocking time and practise using my org-capture template for interrupting task so that I can use C-u with C-c j (my binding for org-clock-goto) to jump to a recently-clocked task.
  • Braindumps can help me use non-computer time to flesh out notes for things I'm working on or ideas for next steps.
  • If I skim the descriptions of new packages in Emacs News (maybe even the READMEs instead of just the one-liners), I'll probably retain a brief sense of what's out there and what things are called.
  • Vector search across package descriptions and function docstrings could be an even more powerful way to discover things that are close to something I want to do.
  • Using elisp-demos to add more examples to functions can help me look up things I frequently use but don't remember.
  • Figuring out more modern IDE features like refactoring support, on-the-fly error checking, and code navigation could help me code faster.

So that's how I tinker with Emacs for fun: start with something that mostly works, keep an eye out for opportunities to make things better, use tinkering as a way to make doing things more fun, look for things that are nearby, and

Footnotes:

1

I used to think this was the Zeigarnik effect, but it turns out the Zeigarnik effect is about remembering incomplete tasks versus completed tasks, while the Ovsiankina effect is more about intrusive thoughts and wanting to get back to that incomplete task.

2

At the moment, she likes Eyecraftmc and J Perm.

View org source for this post