Expanding yasnippets by voice in Emacs and other applications
| emacs, audio, speech-recognitionYasnippet is a template system for Emacs. I want to use it by voice. I'd like to be able to say things like "Okay, define interactive function" and have that expand to a matching snippet in Emacs or other applications. Here's a quick demonstration of expanding simple snippets:
Transcript
- 00:00 So I've defined some yasnippets with names that I can say. Here, for example, in this menu, you can see I've got "define interactive function" and "with a buffer that I'll display." And in fundamental mode, I have some other things too. Let's give it a try.
- 00:19 I press my shortcut. "Okay, define an interactive function." You can see that this is a yasnippet. Tab navigation still works.
- 00:33 I can say, "OK, with a buffer that I'll display," and it expands that also.
- 00:45 I can expand snippets in other applications as well, thanks to a global keyboard shortcut.
- 00:50 Here, for example, I can say, "OK, my email." It inserts my email address.
- 01:02 Yasnippet definitions can also execute Emacs Lisp. So I can say, "OK, date today," and have that evaluated to the actual date.
- 01:21 So that's an example of using voice to expand snippets.
This is handled by the following code:
(defun my-whisper-maybe-expand-snippet (text)
"Add to `whisper-insert-text-at-point'."
(if (and text
(string-match
"^ok\\(?:ay\\)?[,\\.]? \\(.+\\)" text))
(let* ((name
(downcase
(string-trim
(replace-regexp-in-string "[,\\.]" "" (match-string 1 text)))))
(matching
(seq-find (lambda (o)
(subed-word-data-compare-normalized-string-distance
name
(downcase (yas--template-name o))))
(yas--all-templates (yas--get-snippet-tables)))))
(if matching
(progn
(if (frame-focus-state)
(progn
(yas-expand-snippet matching)
nil)
;; In another application
(with-temp-buffer
(yas-minor-mode)
(yas-expand-snippet matching)
(buffer-string))))
text))
text))
This code relies on my fork of whisper.el, which lets me specify a list of functions for whisper-insert-text-at-point. (I haven't asked for upstream review yet because I'm still testing things, and I don't know if it actually works for anyone else yet.) It does approximate matching on the snippet name using a function from subed-word-data.el which just uses string-distance. I could probably duplicate the function in my config, but then I'd have to update it in two places if I come up with more ideas.
The code for inserting into other functions is defined in my-whisper-maybe-type, which is very simple:
(defun my-whisper-maybe-type (text)
"If Emacs is not the focused app, simulate typing TEXT.
Add this function to `whisper-insert-text-at-point'."
(when text
(if (frame-focus-state)
text
(make-process :name "xdotool" :command
(list "xdotool" "type"
text))
nil)))
Someday I'd like to provide alternative names for snippets. I also want to make it easy to fill in snippet fields by voice. I'd love to be able to answer minibuffer questions from yas-choose-value, yas-completing-read, and other functions by voice too. Could be fun!
Related:
