Partly inspired by John Kitchin's video showing how to copy screenshots from his iPad and do optical character recognition so he can use the images and text in Org Mode, I'd like to be able to draw quick notes while I'm thinking through a topic on my computer.

Krita might work, but it's awkward to draw on my tablet PC's screen when it's in laptop mode because of the angle. Flipping it to tablet mode is a bit disruptive.

I can draw on my Supernote, which feels a bit more natural. I have a good workflow for recoloring and renaming exported sketches, but exporting via Dropbox is a little slow since it synchronizes all the folders. The SuperNote has a built-in screen mirroring mode with an MJPEG that I can open in a web browser. Saving it to an image is a little complicated, though. ffmpeg doesn't work with the MJPEG that it streams, and I can't figure out how to get stuff out aside from using a browser. I can work around this by using Puppeteer and getting a screenshot. Here's a NodeJS snippet that saves that screenshot to a file.

#!/usr/bin/env nodejs
# This file is tangled to ~/bin/supernote-screenshot.js from my config at https://sachachua.com/dotemacs
# Usage: supernote-screenshot.js [filename]
# Set SUPERNOTE_URL to the URL.

const process = require('process');
const puppeteer = require('puppeteer');
const url = process.env['SUPERNOTE_URL'] || 'http://192.168.1.221:8080/screencast.mjpeg';
const scale = 0.5;
const delay = 2000;

async function takeSupernoteScreenshot() {
  const browser = await puppeteer.launch({headless: 'new'});
  const page = await browser.newPage();
  await page.setViewport({width: 2808 * scale, height: 3744 * scale, deviceScaleFactor: 1});
  page.goto(url);
  await new Promise((resolve, reject) => setTimeout(resolve, delay));
  let filename = process.argv[2] || 'screenshot.png';
  await page.screenshot({type: 'png', path: filename, fullPage: true});
  await browser.close();
}

takeSupernoteScreenshot();

Then I can call that from Emacs Lisp and run it through my usual screenshot insertion process:

(defun my-org-insert-supernote-from-mirror ()
  "Copy the current image from the SuperNote mirror."
  (interactive)
  (let ((filename (expand-file-name (format-time-string "%Y-%m-%d-%H-%M-%S.png") "~/recordings")))
    (shell-command-to-string (concat "NODE_PATH=/usr/lib/node_modules node ~/bin/supernote-screenshot.js " (shell-quote-argument filename)))
    (shell-command-to-string (concat "~/bin/recolor.py --colors c0c0c0,f6f396 " (shell-quote-argument filename)))
    (call-interactively 'my-org-insert-screenshot)))

Ideas for next steps:

• Add image actions:
  • Annotate the image
  • Crop the image
  • Get the text for the image at point and insert it as a details block
  • Get the text for the image at point and copy it to the kill-ring

Both Google Cloud Vision and SuperNote's new handwriting recognition handle my print fine. Neither handle columns the way I'd like, but to be fair, I'm not really sure how I want columns and wrapping handled anyway. I can always experiment with the standard use-case: one column of text, to export as text (with perhaps the occasional sketch, which I can crop and include).

If I can get the hang of writing my thoughts, then it turns some of those bedtime hours into writing hours. Writing by hand feels slow and linear, but it's better than nothing, and thinking takes most of the time anyway. While speech recognition feels like it might be faster in short bursts, I don't have a lot of "talking to myself" time (aside from sleepy brain dumps), and my workflow for processing audio is still slow and disjointed. I can't type on my phone because then A- will want to be on a screen too. I'm glad e-ink devices are different enough not to trigger her sense of unfairness, although sometimes she does ask if she can do mazes or connect-the-dots. Then I switch to knitting until it's really really time to go to bed.

I'm slowly figuring out my workflows for experimenting with and writing about code. Naturally, that's a little more challenging to write about by hand, but I could draft the context. I can think through life stuff too, and maybe look into saving more notes in my Org files.

I've experimented with handwritten blog posts before. Now that I have a little more time to tweak my workflow and think thoughts, maybe I'll get the hang of them!

It looks like the Supernote's real-time recognition is pretty accurate for my handwriting, getting the text out of multiple pages is pretty straightforward.

Here's the raw TXT output from the Supernote.

Here's what it took to edit it into the first part of this post - just adding line-breaks and fixing up some words:

Source images

[[

If I add more lines between paragraphs when writing, I might be able to skip adding them in the text export.

For comparison, here's the text output from Google Cloud Vision.

Tweaking my handwriting workflow
Both Google Cloud Vision and Super Note's new
handwriting recognition handle my print fine. Neither
handle columns the way I'd like, but to be fair,
I'm not really sure how I want columns and wrapping
handled anyway I can always experiment with the
standard use-case
use-case: One column of text, to export
as Text (with perhaps the occasional sketch, which
can crop and include).
If I can get the hang of writing my thoughts,
then it turns some of those bedtime hours into writi
writing
hours. Writing by hand feels slow and linear, but it's
better than nothing, and thinking takes most of the time
anyway while speech recognition feels like it might be
faster in short bursts, don't have a lot of "talking to
myself" time (aside from sleepy braindumps), and my workflow
for processing audio is still slow and disjointed. I can't
type on my phone because then A- will want to be on

I'm glad e-ink devices are different enough
not to trigger her sense of unfairness, although sometimes
she does ask if she can do mazes or connect-the-dots
a screen too
Then I switch to Knitting until it's really really time to
go to bed.
I'm slowly figuring out my workflows for experimenting
with and writing about code. Naturally, that's a little
more challenging to write about by hand, but I could
draft the context. I can think through life stuff too, and
maybe look into saving more notes in my org files
I've experimented with handwritten blog posts before
Now that I have a little more time to tweak my workflow
and think thoughts, maybe I'll get the hang of them!

I'm leaning towards SuperNote's recognition results for long text, although I don't get access to the confidence data so I'll probably just have to delete the misrecognized text if I include sketches.

I want to make it easier to process the sketchnotes I make on my Supernote. I write IDs of the form yyyy-mm-dd-nn to identify my sketches. To avoid duplicates, I get these IDs from the web-based journaling system I wrote. I've started putting the titles and tags into those journal entries as well so that I can reuse them in scripts. When I export a sketch to PNG and synchronize it, the file appears in my ~/Dropbox/Supernote/EXPORT directory on my laptop. Then it goes through this process:

• I use Google Cloud Vision to detect handwriting so that I can find the ID.
  • I retrieve the matching entry from my journal system and rename the file based on the title and tags.
  • If there's no matching entry, I rename the file based on the ID.
• If there are other tags or references in the sketch, I add those to the filename as well.
• I recolor it based on the tags, so parenting-related posts are a little purple, tech/Emacs-related posts are blue, and things are generally highlighted in yellow otherwise.
• I move it to a directory based on the tags.
  • If it's a private sketch, I move it to the directory for my private sketches.
  • If it's a public sketch, I move it to the directory that will eventually get synchronized to sketches.sachachua.com, and I reload the list of sketches after some delay.

The following code does that processing.

Download supernote-daemon

#!/usr/bin/python3
# -*- mode: python -*-

# (c) 2022-2023 Sacha Chua (sacha@sachachua.com) - MIT License

# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:

# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.


import os
import json
import re
import requests
import time
from dotenv import load_dotenv
# Import the Google Cloud client libraries
from google.cloud import vision
from google.cloud.vision_v1 import AnnotateImageResponse
import sys
sys.path.append("/home/sacha/proj/supernote/")
import recolor   # noqa: E402  # muffles flake8 error about import
load_dotenv()


# Set the folder path where the png files are located
folder_path = '/home/sacha/Dropbox/Supernote/EXPORT/'
public_sketch_dir = '/home/sacha/sync/sketches/'
private_sketch_dir = '/home/sacha/sync/private-sketches/'

# Initialize the Google Cloud Vision client
client = vision.ImageAnnotatorClient()
refresh_counter = 0

def extract_text(client, file):
    json_file = file[:-3] + 'json'
    # TODO Preprocess to keep only black text
    with open(file, 'rb') as image_file:
        content = image_file.read()
    # Convert the png file to a Google Cloud Vision image object
    image = vision.Image(content=content)

    # Extract handwriting from the image using the Google Cloud Vision API
    response = client.document_text_detection(image=image)
    response_json = AnnotateImageResponse.to_json(response)
    json_response = json.loads(response_json)
    # Save the response to a json file with the same name as the png file
    with open(json_file, "w") as f:
        json.dump(json_response, f)


def maybe_rename(file):
    # TODO Match on ID
    json_file = file[:-3] + 'json'
    with open(json_file, 'r') as f:
        data = json.load(f)

    # Extract the text from the json file
    text = data['fullTextAnnotation']['text']

    # Check if the text contains a string matching the regex pattern
    pattern = r'(?<!ref:)[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2}'
    match = re.search(pattern, text)
    if match:
        # Get the matched string
        matched_string = match.group(0)
        new_name = matched_string
        from_zid = get_journal_entry(matched_string).strip()
        if from_zid:
            new_name = matched_string + ' ' + from_zid
        tags = get_tags(new_name, text)
        if tags:
            new_name = new_name + ' ' + tags
        ref = get_references(text)
        if ref:
            new_name = new_name + ' ' + ref
        print('Renaming ' + file + ' to ' + new_name)
        # Rename the png and json files to the matched string
        new_filename = os.path.join(os.path.dirname(file), new_name + '.png')
        rename_set(file, new_filename)
        return new_filename


def get_tags(filename, text):
    tags = re.findall(r'(^|\W)#[ \n\t]+', text)
    return ' '.join(filter(lambda x: x not in filename, tags))


def get_references(text):
    refs = re.findall(r'!ref:[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2}', text)
    return ' '.join(refs)


def get_journal_entry(zid):
    resp = requests.get('https://' + os.environ['JOURNAL_USER']
                        + ':' + os.environ['JOURNAL_PASS']
                        + '@journal.sachachua.com/api/entries/' + zid)
    j = resp.json()
    if j and not re.search('^I thought about', j['Note']):
        return j['Note']


def get_color_map(filename, text=None):
    if text:
        together = filename + ' ' + text
    else:
        together = filename
    if re.search('r#(parenting|purple|life)', together):
        return {'9d9d9d': '8754a1', 'c9c9c9': 'e4c1d9'}  # parenting is purplish
    elif re.search(r'#(emacs|geek|tech|blue)', together):
        return {'9d9d9d': '2b64a9', 'c9c9c9': 'b3e3f1'}  # geeky stuff in light/dark blue
    else:
        return {'9d9d9d': '884636', 'c9c9c9': 'f6f396'}  # yellow highlighter, dark brown


def rename_set(old_name, new_name):
    if old_name != new_name:
        old_json = old_name[:-3] + 'json'
        new_json = new_name[:-3] + 'json'
        os.rename(old_name, new_name)
        os.rename(old_json, new_json)


def recolor_based_on_filename(filename):
    color_map = get_color_map(filename)
    recolored = recolor.map_colors(filename, color_map)
    # possibly rename based on the filename
    new_filename = re.sub(' #(purple|blue)', '', filename)
    rename_set(filename, new_filename)
    recolored.save(new_filename)


def move_processed_sketch(file):
    global refresh_counter
    if '#private' in file:
        output_dir = private_sketch_dir
    elif '#' in file:
        output_dir = public_sketch_dir
        refresh_counter = 3
    else:
        return file
    new_filename = os.path.join(output_dir, os.path.basename(file))
    rename_set(file, new_filename)
    return new_filename


def process_file(file):
    json_file = file[:-3] + 'json'
    # Check if a corresponding json file already exists
    if not os.path.exists(json_file):
        extract_text(client, file)
    if not re.search('[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{2} ', file):
        file = maybe_rename(file)
    recolor_based_on_filename(file)
    move_processed_sketch(file)


def process_dir(folder_path):
    global processed_files
    # Iterate through all png files in the specified folder
    files = sorted(os.listdir(folder_path))
    for file in files:
        if file.endswith('.png') and '_' in file:
            print("Processing ", file)
            process_file(os.path.join(folder_path, file))


def daemon(folder_path, wait):
    global refresh_counter
    while True:
        process_dir(folder_path)
        time.sleep(wait)
        if refresh_counter > 0:
            refresh_counter = refresh_counter - 1
            if refresh_counter == 0:
                print("Reloading sketches")
                requests.get('https://' + os.environ['JOURNAL_USER'] + ':'
                             + os.environ['JOURNAL_PASS']
                             + '@sketches.sachachua.com/reload?python=1')


if __name__ == '__main__':
    # Create a set to store the names of processed files
    processed_files = set()
    if len(sys.argv) > 1:
        if os.path.isdir(sys.argv[1]):
            folder_path = sys.argv[1]
            daemon(folder_path, 300)
        else:
            for f in sys.argv[1:]:
                process_file(f)
    else:
        daemon(folder_path, 300)

It uses this script I wrote to recolor my sketches with Python.

I'm contemplating writing some annotation tools to make it easier to turn the detected text into useful text for searching or writing about because the sketches throw off the recognition (misrecognized text, low confidence) and the columns mess up the line wrapping. Low priority, though.

My handwriting (at least for numbers) is probably simple enough that I might be able to train Tesseract OCR to process that someday. And who knows, maybe some organization will release a pre-trained model for offline handwriting recognition that'll be as useful as OpenAI Whisper is for audio files. That would be neat!

I was thinking about how to prepare for my next 10-year review, since I'll turn 40 this year. I've been writing yearly reviews with some regularity and monthly reviews sporadically, and I figured it would be nice to have those posts in an EPUB so that I can read them on my e-reader and annotate them as I do my review.

I use the 11ty static site generator to publish my blog as HTML files, since I currently can't keep more than Emacs Lisp, Javascript, and Python in my brain. (No Hugo or Jekyll for me at the moment.) I briefly thought about getting 11ty to create that archive for me, but I realized it might be easier to just write it as an external script instead of trying to figure out how to get 11ty to export one thing conditionally.

One of the things I've configured 11ty to make is a JSON file that includes all of my posts with dates, titles, permalinks, and categories. It was easy to then parse this list and filter it to get the posts I wanted. I parsed the HTML out of the _site directory that 11ty produces instead of fetching the pages from my webserver. I got the images from my webserver, though, and I made a local cache and rewrote the URLs. That way, the EPUB conversion could include the images.

Download blog.js

const blog = require('/home/sacha/proj/static-blog/_site/blog/all/index.json');
const cheerio = require('cheerio');
const base = '/home/sacha/proj/static-blog/_site';
const fs = require('fs');
const path = require('path');

function slugify(p) {
  return p.permalink.replace('/blog', 'post-').replace(/\//g, '-');
}

async function processPost(p) {
  console.log('Processing '+ p.permalink);
  let $ = cheerio.load(fs.readFileSync(base + p.permalink + 'index.html'));
  $('#comment').remove();
  let images = $('article img');
  await Promise.all(images.map((i, e) => {
    let url = $(e).attr('src');
    const outputFileName = 'images/' + path.basename(url).replace(/ |%20|%23/g, '-');
    $(e).attr('src', outputFileName);
    $(e).attr('style', 'max-height: 100%; max-width: 100%; ' + ($(e).attr('style') || ''));
    $(e).attr('srcset', null);
    $(e).attr('sizes', null);
    $(e).attr('width', null);
    $(e).attr('height', null);
    if (!fs.existsSync(outputFileName)) {
      console.log('fetch', outputFileName);
      return fetch(url).then(res => res.arrayBuffer()).then(data => {
        const buffer = Buffer.from(data);
        return fs.createWriteStream(outputFileName).write(buffer);
      });
    } else {
      console.log(outputFileName, 'exists');
      return null;
    }
  }));
  console.log('Done ' + p.permalink);
  let slug = slugify(p);
  $('article h2').attr('id', slug);
  let header = $('article header').html();
  let entry = $('article .entry').html();
  return `<article>${header}${entry}</article>`;
}

let last10 = blog.filter((p) => p.date >= '2013-08-01');
let posts = last10.filter((p) => p.categories.indexOf('yearly') >= 0)
    .concat(blog.filter((p) => p.title == 'Turning 30: A review of the last decade'))
    .concat(last10.filter((p) => p.categories.indexOf('monthly') >= 0));

let toc = '<h1>Table of Contents</h1><ul>' + posts.map((p) => {
  return `<li><a href="#${slugify(p)}">${p.title}</a></li>\n`;
}).join('') + '</ul>';

let content = posts.reduce(async (prev, val) => {
    return await prev + await processPost(val);
  }, '');
content.then((data) => {
  fs.writeFileSync('archive.html',
                   `<html><body>${toc}${data}</body></html>`);

});

This created an archive.html with my posts, using the images/ directory for the images. Then I used my shell script for converting and copying files to convert it to EPUB and copy it over.

On the SuperNote, I can highlight text by drawing square brackets around it. If I tap that text, I can write or draw underneath it. Here's what that looks like:

These notes are collected into a "Digest" view, and I can export things from there. (Example: archive.pdf)

(Hmm, maybe I should ask them about hiding the pencil icon…)

Anyway, I think that might be a good starting point for my review.

A- wants me to sit with her at bedtime. She also wants to read a stack of books until she gets sleepy. This means I sometimes have an hour (or even two) of sitting quietly with her which I can use for writing, drawing, reading, or knitting, as long as I'm quiet. ("Mama, keepp your thoughts to yourself!")

My Supernote A5X supports EPUBs and PDFs, but doesn't support HTML files or my library's e-book platform (Libby), and I'm not too keen on the Kindle app. So I need to load it up with my own collection of books, manuals, API documentation, and notes.

Org Mode can export to EPUBs and PDFs well enough. If I make the output file a symbolic link to the same file in the Dropbox folder that's synchronized with my Supernote, I can re-export the EPUB and it will end up in the right place when I sync. I've started accumulating little snippets from the digest of my reading highlights, since putting them into Org Mode allows me to organize them and summarize them in different ways. It feels good to be collecting and organizing things I'm learning.

I plan to use this reading time to skim documentation for interesting things, since sometimes the challenges are more about knowing something exists and what it's called. Then I can copy the digests into my reference.org and export it as an EPUB or PDF, review that periodically, and maybe add some shortcuts to my Emacs configuration so that I can quickly jump to lines in my reference file.

#!/bin/bash
INBOX=~/Dropbox/Supernote/INBOX
for FILE in "$@"; do
    if [[ "$FILE" == *.html ]]; then
        ebook-convert "$FILE" $INBOX/$(basename "$FILE" .html).epub --extra-css 'img { max-width: 100% !important; max-weight: 100% !important }'
        # or pdf: wkhtmltopdf --no-background "$FILE" $INBOX/$(basename "$FILE" .html).pdf
    elif [[ "$FILE" == *.xml ]]; then
        dbtoepub "$FILE" -o $INBOX/$(basename "$FILE" .xml).epub
    elif [[ "$FILE" == *.texi ]]; then
        texi2pdf "$FILE" -o $INBOX/$(basename "$FILE" .texi).pdf
    elif [[ "$FILE" == *.org ]]; then
        emacs -Q --batch "$FILE" --eval "(progn (package-initialize) (use-package 'ox-epub) (org-epub-export-to-epub))"
        cp "${FILE%.*}".epub $INBOX
    else
        cp "$FILE" $INBOX
    fi
done

(defvar my-supernote-inbox "~/Dropbox/Supernote/INBOX")
(defun my-save-manpage-to-supernote (path)
  (interactive (list (woman-file-name nil)))
  (let* ((base (file-name-base path))
         (temp-html (make-temp-file base nil ".html")))
    (with-temp-buffer
      (insert-file-contents path)
      (call-process-region (point-min) (point-max) "man2html" t t)
      (when (re-search-backward "Invalid Man Page" nil t)
        (delete-file temp-html)
        (error "Could not convert."))
      (write-file temp-html))
    (call-process "ebook-convert" nil (get-buffer-create "*temp*") nil temp-html
                  (expand-file-name (concat base ".epub") my-supernote-inbox))
    (delete-file temp-html)))

Welcome to autokey's documentation!
===================================

   .. autoclass:: autokey.scripting.Keyboard
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Mouse
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Store
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.QtDialog
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.System
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.QtClipboard
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Window
      :members:
      :undoc-members:

   .. autoclass:: autokey.scripting.Engine
      :members:
      :undoc-members:

The SuperNote lets me draw with black, dark gray (0x9d), gray (0xc9), or white. I wanted to make it easy to recolor them, since a little splash of colour makes sketches more fun and also makes them easier to pick out from thumbnails. Here's the Python script I wrote:

Download recolor.py

#!/usr/bin/python3
# Recolor PNGs
#
# (c) 2022 Sacha Chua (sacha@sachachua.com) - MIT License
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:

# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

import numpy as np
import os
import csv
import argparse
from PIL import Image

DARK_GRAY = 0x9d
GRAY = 0xc9
WHITE = 0xfe

color_dict = {}

def color_to_tuple(color_dict, s):
    if s in color_dict:
        s = color_dict[s]
    s = s.lstrip('#')
    if (s == '.'):
        return (None, None, None)
    elif (len(s) == 2):
        return (int(s, 16), int(s, 16), int(s, 16))
    else:
        return tuple(int(s[i:i + 2], 16) for i in (0, 2, 4))

def load_color_dict(filename):
    dict = {}
    with open(os.path.expanduser(filename), newline='') as csvfile:
        reader = csv.reader(csvfile, delimiter=',', quotechar='"')
        for row in reader:
            dict[row[0]] = row[1]
    return dict


def map_colors(input, color_map):
    if isinstance(input, str):
        im = Image.open(input).convert('RGB')
    else:
        im = input
    data = np.array(im)
    red, green, blue = data[:, :, 0], data[:, :, 1], data[:, :, 2]
    for from_c, to_c in color_map.items():
        from_r, from_b, from_g = color_to_tuple(color_dict, from_c)
        to_r, to_b, to_g = color_to_tuple(color_dict, to_c)
        mask = (red == from_r) & (green == from_g) & (blue == from_b)
        data[:, :, :3][mask] = [to_r, to_b, to_g]
    return Image.fromarray(data)

def set_colors_by_freq(input, color_list):
    if isinstance(input, str):
        im = Image.open(input).convert('RGB')
    else:
        im = input
    data = np.array(im)
    red, green, blue = data[:, :, 0], data[:, :, 1], data[:, :, 2]
    sorted_colors = get_colors_by_freq(input)
    freq = iter(color_list.split(','))
    for i, f in enumerate(freq):
        if f != '.':
            to_r, to_b, to_g = color_to_tuple(color_dict, f)
            by_freq = sorted_colors[i][1]
            if isinstance(by_freq, np.uint8):
                mask = (red == by_freq) & (green == by_freq) & (blue == by_freq)
            else:
                mask = (red == by_freq[0]) & (green == by_freq[1]) & (blue == by_freq[2])
            data[:, :, :3][mask] = [to_r, to_b, to_g]
    return Image.fromarray(data)

def color_string_to_map(s):
    color_map = {}
    colors = iter(args.colors.split(','))
    for from_c in colors:
        to_c = next(colors)
        color_map[from_c] = to_c
    return color_map

def get_colors_by_freq(input):
    if isinstance(input, str):
        im = Image.open(input).convert('RGB')
    else:
        im = input
    colors = im.getcolors(im.size[0] * im.size[1])
    return sorted(colors, key=lambda x: x[0], reverse=True)


def print_colors(input):
    sorted_colors = get_colors_by_freq(input)
    for x in sorted_colors:
        if x[0] > 10:
            if isinstance(x[1], np.uint8):
                print('%02x %d' % (x[1], x[0]))
            else:
                print(''.join(['%02x' % c for c in x[1]]) + ' %d' % x[0])


def process_file(input):
    print(input)
    if args.preview:
        output = None
    else:
        output = args.output if args.output else input
        if os.path.isdir(output):
            output = os.path.join(output, os.path.basename(input))
    im = Image.open(input).convert('RGB')
    if args.colors:
        im = map_colors(im, color_string_to_map(args.colors))
    elif args.freq:
        im = set_colors_by_freq(im, args.freq)
    else:
        print_colors(im)
        exit(0)
    if args.preview:
        im.thumbnail((700, 700), Image.ANTIALIAS)
        im.show()
    elif output:
        im.save(output)


if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description='Recolor a PNG.',
        formatter_class=argparse.RawTextHelpFormatter,
        epilog="If neither --colors nor --freq are specified, "
        + "display the most frequent colours in the image.")
    parser.add_argument('--colors', help="""Comma-separated list of RGB hex values in the form of old,new,old,new
    Examples:
    9d,ffaaaa,c9,ffd2d2 - reddish
    c9,ffea96 - yellow highlighter
    c9,d2d2ff - light blue
    """)
    parser.add_argument('--freq', help="Color replacements in order of descending frequency (ex: .,ffea96). .: use original color")
    parser.add_argument('--csv', help="CSV of color names to use in the form of colorname,hex")
    parser.add_argument('--preview', help="Preview only", action='store_const', const=True)
    parser.add_argument('input', nargs="+", help="Input file")
    parser.add_argument('--output', help="Output file. If not specified, overwrite input file.")

    args = parser.parse_args()
    color_dict = load_color_dict(args.csv) if args.csv else {}
    for input in args.input:
        process_file(os.path.join(os.getcwd(), input))

I don't think in hex colours, so I added a way to refer to colours by names. I converted this list of Copic CSS colours to a CSV by copying the text, pasting it into a file, and doing a little replacement. It's not complete, but I can copy selected colours from this longer list. I can also add my own. The CSV looks a little like this:

lightgreen,cfe8d3
lightyellow,f6f396
lightblue,b3e3f1
y02,f6f396
w2,ddddd5
b02,b3e3f1
...

It doesn't do any fuzzing or clustering of similar colours, so it won't work well on antialiased images. For the simple sketches I make with the SuperNote, though, it seems to work well enough.

I can preview my changes with something like ./recolor.py ~/sketches/"2022-08-02-01 Playing with my drawing workflow #supernote #drawing #workflow #sketching #kaizen.png" --csv colors.csv --freq .,lightyellow --preview , and then I can take the --preview flag off to overwrite the PNG.

Here's what the output looks like:

I've had my SuperNote A5X for a month now, and I really like it.

I put my visual book notes and visual library notes into a Dropbox shared folder so that you can check them out if you have a Supernote. If you don't have a Supernote, you can find my visual book notes at sketches.sachachua.com. Enjoy!