Kiibohd Engineering

Juicy technical updates from the engineers at Input Club.

2018-03-17

Engineering Update March 2018

2018-03-17

Engineering Update March 2018

by Input Club

Here at Input Club we tend to do a lot more than what gets publicized in our product and Kickstarter updates. And while all our code and designs are opensource and up on GitHub it can be overwhelming (even for engineers!) to figure out what we’ve actually been working on. So the engineering team here at Input Club is going to start publishing monthly updates on what we’ve been working on!

First off, introductions of the team are in order.

HaaTa - Jacob Alexander

Before turning this into a biography, let’s just say HaaTa has too many projects :laughing:. He works on a bit of everything at Input Club. Inventor of KLL, main firmware engineer for Input Club, keyboard collector, measurer of force curves, Hako/Halo switch inventor and designer of keyboards at Input Club. He’s been in the keyboard community since 2009 and is always excited to talk about keyboards.

Parak - Gennadiy Nerubayev

At Input Club, Parak is responsible for PCB design and all-around KiCad wizard. He keeps PCB fabs on their toes with his stringent manufacturing checklist (we’ve qualified a lot more pcb fabs than we’d care to admit here at Input Club :sweat_smile:). Parak, the ebay master responsible for most of HaaTa’s keyboard collection, is a proficient collector in tools of great quality (such as IBM keyboards) in his own right.

Over^Kill - Brandon Muzzin

Ever wonder how Input Club makes their simple, but beautiful, cases. By going Over^Kill, that’s how :stuck_out_tongue:. Over^Kill deftly designs the cases, not just to look good (as an engineer), but designed to be manufactured to keep the quality high and the failures low (because he’s a manufacturing wizard). Lately he’s been really into using IronCAD. Over^Kill also helped turn HaaTa’s proposed force curve in the final Hako and Halo switches.

jbondeson - Jeremy Bondeson

Master of the configurator and all things related to the web, jbondeson is a connoisseur of keycaps. He has all sorts of cool things planned out for the desktop configurator all in the name of R, G and B. In his quest for quality, he rewrites code about just as much as HaaTa does :metal:.

smasher816 - Rowan Decker

While the most recent addition to the Input Club team, smasher816 has been helping out since before the Infinity 60% first shipped out in 2015 :cat:. Proficient in both PCB and firmware design.

Git Repos

Next up, git repos! Open Source is very important to Input Club :tropical_drink:. It likens back to a day where you got a service manual with your keyboard (1970s IBM) and you could just lookup the part :memo: that was not working correctly. Or better yet, figure out how to build your own!

Unless otherwise specified, we use GPLv3 for software and the CERN OHL v1.2 for things we work on. Or if we’re contributing back to another project, we maintain that license (and usually try to upstream changes).

All the Input Club git repos can be found on GitHub.

controller.git

The controller git repo contains all the code that goes onto the keyboard as firmware. For those that are not familiar, firmware is compiled software that is stored on a non-volatile device (like inside a keyboard’s flash storage). This repo contains both the :truck: driver code for various keyboard MCUs (microcontrollers) as well as the implementation of KLL. This is the repo you should be watching if you’re interested in new keyboard features and bug-fixes.

The code here is mainly written in C with a CMake build system.

Both Travis-CI and Appveyor are used to make sure the firmware can be compiled for Windows (Cygwin/Bash), Linux and macOS. The recommended way to compile the firmware manually is through Docker.

Some fun facts

kll.git

This repo contains the KLL compiler source code. KLL is the DSL that HaaTa wrote to define keyboard functionality in a distinct and widely compatible way. It uses a powerful :muscle: Trigger:Result mapping idiom design to smoothly compile down into code that can be compiled for a microcontroller.

The compiler is written in Python and is based on funcparserlib.

The KLL compiler supports multiple emitters, so while it currently supports kiibohd, KLL (yes it can regenerate back to a KLL file) and none (syntax checking), it is possible to add KLL support to other keyboard firmwares just by adding a new emitter.

Travis-CI is used to test the KLL compiler against a variety of Python 3 versions.

Some fun facts

kll-spec.git

The kll-spec repo contains the KLL spec (written in LaTex). If you’re interested in seriously understanding KLL you’ll definitely want to read the spec. So far only 0.3d is fully implemented; however, much of 0.5 has been added (i.e. LED support). The major feature left to implement in KLL 0.5 is state-scheduling (e.g. tap-keys, short/long presses).

Overleaf was used to compile/maintain the docs in the past. Unfortunately there have been some issues :cry: lately so all compilation hosting is done on GitHub now (using Travis-CI).

Some fun facts

configurator.kll

This is the :fire: awesome desktop app that jbondeson has been working on. It was designed to handle both layout mapping as well as LED configuration for you keyboard. Rather than attempting to bundle a compilation environment, the configurator uses a KiiConf server to handle firmware generation.

Written mainly in Clojure.

Both Travis-CI and Appveyor are used to automate the release generation. All of the releases (including release notes) can be found on GitHub.

KiiConf.git

Originally started by :shipit: matt3o for the WhiteFox keyboard, KiiConf is the web-based configuration utility for our keyboards. Using your configuration it generates a KLL file. Which the configurator backend then uses to generate a firmware bundle.

Written mainly in JavaScript but also contains quite a bit of PHP and Shell code as well.

Travis-CI is used, in combination with docker, to make sure releases keep on working!

kiidrv.git

This is a relatively new repo forked from libwdi by smasher816. It contains the code for the upcoming automated Zadig driver installation (no more messing with Zadig!).

Written in C and C++.

Appveyor is used to generated automated builds.

Do remember that for drivers Win32 vs. x64 really does matter.

case.git

The cases of Input Club are :triangular_ruler: designed by Over^Kill using IronCAD. We generally opensource (CERN OHL) the design around 30 days (usually because we’re really busy :horse_racing: trying to ship the keyboard, haha) after the first units of a design ships. There are a few reasons for this, though the most important is to make sure we release files that are the same as what ships.

The files can be opened using any 3d model/cad program that can work with .step files (e.g. FreeCAD).

pcb.git

Input Club’s pcb are :triangular_ruler: designed by Parak using KiCad. We opensource (CERN OHL) the schematic, layout, BOM (bill of materials) as well as the :cake: gerbers used to manufacture the pcbs. These are generally released around the same time as the case. Historically we generally revise the pcb between runs more often than the case.

KiCad can be used to view the schematics, layout and gerbers (though we recommend gerbv on Windows and Linux for viewing gerbers).

dfu-util.git

A small :fork_and_knife: fork of dfu-util to maintain a macOS binary release of dfu-util.

Written in C.

Uses Travis-CI to publish releases.

kii-dfu.git

This is the :older_man: old Windows GUI front-end for dfu-util. It has been long deprecated, please use the configurator instead. While it also works for macOS and Linux it is tricky to package correctly for all distributions.

Written in C++ and Qt5.

manufacturing.git

This repository contains both some scripts used in keyboard :construction_worker: manufacturing (bootloader flashing) as well as a detailed wiki on the tools and processes needed to flash and validate shipping keyboards. The wiki also has details on how to build a bootloader flashing adapter which is useful if you’re trying to manufacture your own pcb.

If you’re interested in how we manufacture keyboards, definitely reach out to HaaTa on Discord.

programmer.git

A fork of the :sheep: McHCK SWD flashing utility for the BusPirate. Contains some small updates for the Kinetis mk20dx256vlh7 MCU as well as some additional debugging used in manufacturing.

This isn’t needed unless you’re looking at flashing the bootloader on your keyboard with a BusPirate in SWD mode. Also, don’t worry, you don’t need a bootloader update (you’re not missing out on any keyboard features).

Written in Ruby.

openocd.git

This is a small fork of :octopus: OpenOCD which contains patches for using openocd with a BusPirate and the SWD protocol. The BusPirate and OpenOCD are used in manufacturing, though we use JTAG if possible (same cable, much faster but is not supported by all MCUs).

Written mainly in C and TCL.

force.git

While not particularly useful for most of you, this repo contains the firmware source as well as algorithms used for :ram: HaaTa’s force gauge.

It also contains the code which publishes the graph data onto Plotly.

Written mainly in C (firmware) and Python (algorithms and web interaction).

BOSSA.git

This is a relatively new repository fork. :dancer: BOSSA is a tool used to flash MCUs with the SAM-BA integrated bootloader which will be used for manufacturing of SAM4S keyboards. It has GUI and command line releases for Windows, Linux and macOS.

Written mainly in C++.

Recent Development

Configurator

The latest version of the configurator (it’s auto-updating! :v:) can always be found on GitHub. Our most recent release is v0.4.1.

The configurator is comprised of two parts, the UI (the configurator download) and the KiiConf server (which handles generating the firmware image). We update these two pieces separately as we qualify stable firmware for the configurator rather than just releasing the latest (and possibly unstable) firmware.

Manufacturing before :confetti_ball: Chinese Lunar New Year has put a damper on configurator development, but it should start to ramp-up again soon. The next two new features to help simplify firmware downloading.

The first would be, auto-downloading of dfu-util binaries. Not all computers have a working version of dfu-util available so if the configurator can’t find one on the system the configurator will download a known working dfu-util for your system. This is particularly helpful for both Windows and macOS.

The second is automating the Zadig driver installation. Using the new kiidrv project by smasher816, we can automatically determine whether you need to install the Zadig driver and handle it for you. No more accidentally installing the driver on the wrong interfaces!

Firmware

We’ve been hard at work on the keyboard firmware since the K-Type launched working on :bug: bugs and adding new :full_moon_with_face: features.

One of the major improvements to the firmware was moving from a single execution loop to two concurrent threads. Prior to the K-Type, the only heavy lifting the MCU had to do was scan keys, send updates to USB and occasionally update some peripherals (e.g. LCD screen, change LED :high_brightness: brightness, etc.). However, to fully utilize the RGB’ness of the K-Type a high frame rate required using the majority of the CPU time servicing animations…instead of scanning for keypresses.

After thinking about the problem for a while, HaaTa decided to rewrite a large portion of the firmware and split execution into two different paths. You can think of it as two different threads: one that runs consistently and reliably for key scanning and the other as :dash: fast as it can to deal with animations. Periodic and polling, respectively.

To prevent flickering, each LED frame must be serviced as quickly as possible. Which means you need to spend as little time away from the LED processing as possible. And, instead of scanning all the keys, then processing all the LEDs, just process one strobe (matrix column) of keys at a time. Then process one frame of LEDs, then onto the next strobe.

For those that are curious, the periodic cli command will let you adjust, in real time, how many cycles to wait between periodic scans.

Work has been done to fix issues with KLL macros as well as some offset errors which were causing some keyboards to reset (yep, off by 1 bug). These were all fixed using the new the :scroll: Self-Testing KLL build feature.

The other large project going on right now is preparing the firmware for Kira. This will be the first keyboard design since 2015 to use a new MCU! More SRAM for code, more flash for animations, more :japanese_ogre: Hz for spamming USB packets over the interwebs. All-in-all, just better. However, we’re moving away from NXP’s (originally Freescale’s) Kinetis K20 line and onto Microchip’s (originally Atmel’s) SAM4S line of MCUs. Also, before you say anything :speak_no_evil:, no, this isn’t an AVR, it’s a full blown ARM Cortex-M4, just like the Kinetis K20 series.

KLL Compiler

Recently there have been a lot of changes in the KLL compiler in regards to JSON output. JSON is used by both the configurator/KiiConf as well as host-side KLL to gather more information that is available to the KLL compiler, but not the final output generated by the KLL compiler. This includes things such as all defined KLL capabilities, per-layer trigger:result pairs and physical keymapping.

Another recent change was classifying some capabilities as :guardsman: thread-safe. With the recent change in the controller firmware to use 2 threads of execution (periodic and poll) some capabilities may need to access resources that are not thread-safe. Capabilities that are thread-safe may be called immediately inside the periodic thread, while non-thread-safe capabilities must be queued up for the polling thread.

KLL Spec

As we’re still trying to catch-up and finish the KLL 0.5 support, there hasn’t been a lot of action on the KLL spec. However, there are a few things that will likely sneak into the KLL 0.5 spec before it is complete. Something that will make it for KLL 0.5 are lock LED triggers. These are useful for triggering off of things such as :see_no_evil: CapsLock and NumLock.

Recent Projects

kiidrv

If you use one of our keyboards on Windows, you probably recognize this.

And if you’ve been unlucky :hear_no_evil:, you’ve probably messed up and had to uninstall the driver as well (which is a bit tricky). Zadig is a great tool, but why can’t it just be done for you (like on macOS and Linux) and just work. That’s the goal of kiidrv, to make flashing your keyboard on Windows more seamless.

smasher816 integrated both libwdi, the library used to create Zadig and devcon, an API to work with the Device Manager, to create kiidrv. This means we can not only install the correct driver for your keyboard, but also validate that you’ve only installed it correctly (in case you used Zadig on the wrong device by :hurtrealbad: mistake; really easy to do btw).

Feel free to use kiidrv in your own projects :smiley_cat: as it’s fully opensource under GPLv3 and LGPLv3, and just a useful tool in general. Make sure to download the correct architecture for your version of Windows.

Self-Testing KLL

For a long time HaaTa has had a difficult time :rage1: testing KLL. By design, KLL supports an enormous number of configurations :rage3: multiplied by the number of keyboards that support it. Sadly, this means it’s also :rage4: impossible to test every configuration ahead of time.

But, KLL has a very :suspect: interesting design aspect to it. The trigger:result pairs that define what you pressed and what should happen when you press. Or said a different way, both input and outputs are known ahead of time :godmode:! This means KLL defines what we’re expecting to happen, which for those software engineers out there means that this is actually a constrained problem that can be unit tested.

# Pressing key, 0x10, should result in A
S0x10 : U"a";

# Pressing key, 0x24, should result in B
S0x24 : U"b";

# Pressing abc, should result in xyz
'abs' : 'xyz';

So, back in 2016 HaaTa began his port of the KLL firmware to :computer: x86. Instead of setting up a complicated hardware setup, the KLL firmware implementation relies entirely on CPU instructions, so no emulation is required for the trigger:result macro logic. Then, to support all of the hardware specific calls, Python callbacks were implemented with Scan and Output. The goal of this shared library :green_book: (kiibohd.so) was to be able to control the KLL processing :bicyclist: cycle-by-cycle in order to construct any sort of situation that may occur on a keyboard. Even the ones that are basically impossible to trigger on a physical keyboard (e.g. press 10 keys in a row exactly 10 us apart for 3 ms each).

The tests not only require interaction with the newly generated kiibohd.so, but also what the input and outputs of trigger:result pairs are. Instead of re-parsing the KLL files, the KLL compiler generates a kll.json file which contains detailed information about the compiled KLL layout. With this information a test can be generated for each possible key combination on each layer and what the result of each test should be.

Inputting trigger information is fairly straightforward as each trigger currently defines a scancode. This scancode is then sent as a pressed scancode. Next, the library is indicated it may process a single loop. Before checking the result, the trigger cleanup must begin. The trigger scancode is released, as to simulate an extremely quick press/release on a keyboard.

Validating :microscope: the result is a bit trickier as results are actually KLL capabilities (i.e. C functions). Some of these functions, such as USB, will output a USB code that is easy to validate as a press/release event. However, some capabilities trigger hardware that is not available to the shared library. In this case a capability history buffer is maintained. This history buffer is compared to what was expected, using the kll.json information.

While this is only a start, self-testing KLL should be working for all macros (sequence and combo) currently supported by the firmware.

Now, it’s possible to :tada: self-test your own KLL files. While this functionality is currently limited to compiling the firmware yourself, it will be added to the configurator as well in the future. Not only does it give you an indication on which expressions are having issues, it also provides developers with more information on what the problem actually is (the trickiest part when reporting a bug is providing enough relevant and detailed information to actually solve the issue).

Really though, all this work was done in order to :door: prepare for state scheduling testing (also known as short/long press or tap keys). State scheduling provides so many customization options and timing options that are just not possible to test reliably on a physical keyboard. So we’re now one more step closer to achieving full KLL 0.5 support.

Some useful reading

If you’re curious as to what the output looks like, take a look at this Travis-CI job.

Kira Status

We’re :construction: hard-at-work on both the next prototype case and pcb revisions (with big news in less than a week!). Much of the work on the case has been figuring out how to injection mold the bottom foot. The pcb has so many components on it that it has been tricky to place posts for screw holes. This is extra tricky due to the hotswap pcb needing to be braced against the steel plate without having any switches inserted.

The pcb is our first :bridge_at_night: design using the SAM4S so we’re using all the experience we gained designing the K-Type and Type C WhiteFox to make the Kira even better. Something that people tend to forget, is why old keyboards had bezels. Keyboards used to have big bezels because you could place your keyboard controller chips in those areas. It was a simpler time then. But now, with our :bullettrain_front: sleek and streamlined keyboards, with no arrow cluster or spaced function row, there are very few places to put components on the pcb that won’t interfere with the keyboard switches. Fortunately, there’s a lot of room under the spacebar and a bit near the shift keys.

One cool feature of the SAM4S is the :doughnut: built-in bootloader SAM-BA. While we’ll still be using a custom dfu bootloader to handle layout flashing, this built-in bootloader means that we’ll no longer need an external tool to update to bootloader. But don’t worry, you’ll have to fully open up the case and follow some special instructions (won’t be labeled) in order to activate the built-in bootloader.

Upcoming Projects

And to finish off, some things to look forward to :shaved_ice: in the coming months.

Final Notes

And that’s it for this update! We’re going to try and put out at least one update ever 1-2 months (though maybe not quite this long :sleeping:).

tags: update - engineering - keyboards - firmware - case - pcb - configurator - fun