Rendering Interactive Graphics in Kitty

Using kitty's graphics protocol, one can quickly build visualizations and interactive demos.

Implementation

Usage

gradient-test.cpp

#include <stdio.h>
#include <time.h>

#include "graphical-term-util.h"

int
main() {
  GraphicalTermState *state = (GraphicalTermState *)malloc(sizeof(GraphicalTermState));
  GraphicalTermStart(state);
  GraphicalTermHideTextualCursor(state);

  bool done = false;
  while (!done) {
    GraphicalTermFrameBegin(state);
    if (GraphicalTermIsKeyDown(state, GTKEY_ESCAPE)) {
      done = true;
    }

    // only clear + draw when there is an active framebuffer
    if (state->framebufferPending) {
      // Clear
      int totalPixels = state->framebuffer.width * state->framebuffer.height;
      for (int i = 0; i < totalPixels; i++) {
        state->framebuffer.ptr[i] = 0xFFFF00FF;
      }

      GraphicalTermMoveCursorHome();

      fprintf(stderr,
              "mouse (%u, %u) buttons(%u) size(%u, %u)",
              state->mouse.x,
              state->mouse.y,
              state->mouse.buttons,
              state->framebuffer.width,
              state->framebuffer.height);

      // Draw a red/green gradient across the entire framebuffer
      for (int y = 0; y < state->framebuffer.height; y++) {
        int yoff = y * state->framebuffer.width;
        for (int x = 0; x < state->framebuffer.width; x++) {
          int red = int(float(x) / float(state->framebuffer.width) * 255.0f);
          int green = int(float(y) / float(state->framebuffer.height) * 255.0f);
          state->framebuffer.ptr[yoff + x] = 0xFF000000 | (red & 0xFF) |
                                             ((green & 0xFF) << 8);
        }
      }
    }

    GraphicalTermFrameEnd(state);
  }

  GraphicalTermStop(state);
  free(state);
  return 0;
}

Running this via the approach seen in Using C/C++ as a scripting language (Part 1)

c gradient-test.cpp

Inspiration

twitter/@thingskatedid has a great FAQ describing how to use kitty icat or similar to output graphviz diagrams as images - really cool stuff!
michaeljclark/glkitty - uses OpenGL to render into kitty

Issues

This POC only works on Linux and basically requires kitty to function. Other terminals such as wezterm and alacritty can display images, but they don't have full SGR-pixel mouse support and/or don't support the full kitty keyboard progressive enhancement protocol.

What this means in practice is: this is not a great foundation to build an application that you intend on distributing. I'd limit this to local one-offs, quick proofs of concept, and visualizations.

Running Kitty Under WSL 2.0

using Ubuntu-22.04.2 LTS from the Windows app store

I'm on NVidia which requires the Cuda toolkit for some reason!

sudo apt install libwayland-cursor0 libwayland-egl1 cuda-toolkit-11-0
kitty&

kitty is very out of date on apt so I'd recommend following the install instructions here

Extra

Here are a few more videos showing what you can do with a framebuffer and mouse+keyboard inputs