Mars Rover Robot

This is an ongoing multi-year project.  I need to write this up properly, but I’ve got so much stuff that I’m going to attack this in stages.

Current Status

14 motors, controlled by a raspberry pi.  Arm doesn’t quite work yet.  Everything else works.

Front end – I wrote this in ReactJS, and it communicates with the robot via a websocket to RobotOS, using ‘rosbridge’.  Sorry it’s so pink – it uses infrared LEDs to work at night, but they are kinda overpowering.  Also sorry for the mess – I have two children…

React frontend

In the top left, the green circle is a ‘nipple’ – to let you move the rover about via webpage. Either via mouse or finger press.

In the top right, the xbox controller image shows what buttons are being pressed on an xbox controller, to control the robot. The xbox controller can either be connected to the rover, or connected to the PC / mobile phone – the webpage relays controller commands through to the rover.

Beneath the xbox controller is the list of running processes (“ros nodes”), with cpu and mem information.

Below that is the error messages etc.

Console UI

I’m strong believer in trying to make my projects ‘transparent’ – as in, when you connect, it should be obvious what is going on, and how to do things.

With that in mind, I always create nice colored scripts that show the current status.

Below is the output when I ssh into the raspberry pi.  It shows:

– Wifi status and IP address
– Currently running ROS Node processes, and their mem and cpu status.  Colored Green for good, Red for stopped/crashed.
– Show that ‘rosmon’ is running on ‘screen’ and how to connect to it.
– The command line shows the current git status

rover_putty

I put this script in git in two parts – the first is a bash script to source from bash, and the second is a python script to use ros to get the current status.

See those links for the source code.  I think that every ROS project can benefit from these.

Boogie Rocker Design

Ultrasonic Sensor

I played about with making a robot face.  I combined a Ultrasonic Sensor (HC-SR04) and NeoPixel leds.  The leds reflect the distance – one pixel per 10 of cm.

I’m wondering if I can do SLAM – Simultaneous Location And Mapping  with a single (or a few) very-cheap ultrasonic sensor. Well, the answer is almost certainly no, but I’m very curious to see how far I can get.

NOTE: This is an old design – from more than two years ago

face2

Interactive dialogue on a webpage with React and Promises

Here’s the goal:

A way to make a webpage that lets you have an interactive dialogue by choosing options, like the old Interactive Fiction games.

The input file is very simple:

PRINT Hello!
WAIT ; This is a comment.  Wait for a key press or mouse click
PRINT I would like to ask you a question.
PRINTW Please don't be nervous. ; The 'W' means WAIT afterwards
PRINT Are you happy?
PRINT [0] Yes
PRINT [1] Not really...
CALL INPUTINT(0, 1)
IF RESULT == 0
    PRINTW I'M SO HAPPY THAT YOU'RE HAPPY!
ELSE
    PRINTW Then I am miserable 😦
ENDIF

The challenge is to make a webpage that could read that input, and run it, producing the interactive output shown in the video above.

Perhaps have a think if you don’t know how you would implement this. It is perhaps not so obvious.

Here is the approach that I decided to take.  Note that I only wanted to spend a weekend on this, so I took various shortcuts that I wouldn’t recommend if writing for production:

  1. Read in the input file in, using javascript with a Tokenizer and Parser.
  2. Output javascript using the above Parser, effectively making a transpiler from the input language to javascript.  Call outside javascript functions to ‘PRINT’ etc.
  3. Use ‘eval’ to run the resulting transpiled code.  Yes, eval is evil and this is a case of don’t do what I do.
  4. Use Promises, via async and await, to ‘pause’ and allow interaction.  Implement ‘WAIT’, ‘PRINTW’ ‘INPUTINT’ etc functions in javascript with Promises, so that they only resolve when the user has clicked, typed a number etc.
  5. Display output by just appending to a list, and displaying that list in React.

1. Read the file in, using Javascript with a Tokenizer and Parser

I used jison.  Although the README blurb says that it is for Context Free Grammars, because it is based on BISON which is likewise, it does actually support context.  This was vital because the input file language is not actually a context free grammar.

2. Output javascript using the above Parser, effectively making a transpiler from the input language to javascript

The correct way to do this would be to create an Abstract Syntax Tree, however I didn’t want to take too long on this, and instead simply outputted javascript code as a string.

3. Use ‘eval’ to run the resulting transpiled code.

This is very frowned upon, but this was a weekend project, so….

There is one trick that I used here.  I wrap the entire program to be ‘eval’ed like:

async function() { ..... }()

This allows the program code inside to use async and await to wait for input, and the eval is returning a Promise.  One minor point – when we use eval to evaluate this, we want to catch errors that the Promise throws, to provide clear feedback to the user if there are problems.  E.g.

try {
    await eval(program);
} catch(e) { ... }

4. Use Promises, via async and await, to ‘pause’ and allow interaction.  Implement ‘WAIT’, ‘PRINTW’ ‘INPUTINT’ etc functions in javascript with Promises, so that they only resolve when the user has clicked, typed a number etc.

I used two layers of callbacks, to implement a poor-man’s publish and subscribe system.

So the transpiler turns:

PRINTW Hello!

into:

printLine("Hello!");
await wait();

And the wait() function is implemented as:

async function wait() {
  await new Promise(
    resolve => cb_setClickListener(() => resolve())
  )
}

So we subscribe to a click listener via the callback ‘cb_setClickListener’ and then resolve the promise (and thus resume running the program) when the click is published.

Inside the React page, we now listen for clicks and publish it to the callback:

 this.state.clickListener &&
        this.state.clickListener()
}>

And likewise for keypresses.  (Note, I’ve simplified the code here a bit.  In the real code, I pass the keypressed etc, so that INPUTINT can listen to a particular key).

5. Display output by just appending to a list, and displaying that list in React.

The ‘printLine’ function was implemented like:

function printLine(str) {
  const newLine =
<div>{str}</div>
;
  this.setState({displayLines: [...displayLines, newLine]})
}

One extra detail – if the string starts with a number in a bracket like: “[0] Yes”, then I output a link that publishes that number like:

<div> this.state.keyPressedListener &amp;&amp;
          this.state.keyPressedListener(number)
  }
&gt;{str}</div>

This way, when presented with a choice, the user can just click a link instead.   I maintain two separate output lists, so that I can disable the links once we are done with them.

Conclusion and notes

It worked very nicely! I further extended this to support variables, entering strings, showing pictures, and so on.

The language is actually a real language, but very niche and used almost exclusively by Japanese. It hasn’t seen much acceptance or use outside of Japan.

TypeScript + lodash map and filter

I love TypeScript.  I use it whenever I can.  That said, sometimes it can be…  interesting.  Today, out of the blue, I got the typescript error in code that used to work:

[06:53:30]  typescript: src/mycode.ts, line: 57 
            Property 'video' does not exist on type 'number | (<U>(callbackfn: (value: Page, index: number, 
            array: Page[]) => U, thisA...'. Property 'video' does not exist on type 'number'. 

 

The code looks like:

return _.chain(pages)
        .filter((s, sIdx) => s.video || s.videoEmbedded)
        .map((s, sIdx) => {
            if (s.video) { ... }

Can you spot the ‘error’?

The problem is that s.video || s.videoEmbedded isn’t returning a boolean. It’s return a truthy value, but not a boolean. And the lodash typescript developers made a change 1 month ago that meant that filter() would only accept booleans, not any truthy value. And the lodash typescript developers are finding that fixing this becomes very complicated and complex. See the full conversation here:

https://github.com/DefinitelyTyped/DefinitelyTyped/issues/21485

(Open issue at time of writing. Please leave me feedback or message me if you see this bug get resolved)

The workaround/fix is to just make sure it’s a boolean. E.g. use !! or Boolean(..) or:

return _.chain(pages)
        .filter((s, sIdx) => s.video !== null || s.videoEmbedded !== null )
        .map((s, sIdx) => {
            if (s.video) { ... }

Changing the color of image in HTML with an SVG feColorMatrix filter

In a previous post, I changed the color of a simple image (of a pair of eyes) by converting it to an SVG first, and then changing the SVG colors.

But what if you want to a take a complex image and change the color?

 

In Photoshop/Gimp this can be achieved by creating a new layer on top of the image and filling it with a solid color, and then setting its Mode to ‘Multiply’.  But how can we reproduce this on the web?

There are various solutions using svg filters (feFlood and feBlend) but these are not very well supported in browsers.  So I’ve come up with a solution that is very well supported in all modern browsers, including IE.









...


Replace the numbers 0.5 with the rgb values of the color that you want.  For example, in react:


hexToRgb(hex) {
var result = /^#?([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})$/i.exec(hex);
return result ? {
r: parseInt(result[1], 16),
g: parseInt(result[2], 16),
b: parseInt(result[3], 16)
} : {r: 255, g: 255, b: 255};
}

skinColorDef(colorAsString) {
const hex = hexToRgb(colorAsString); /* <-- See next for an improvement*/
const r = hex.r/255;
const g = hex.g/255;
const b = hex.b/255;
return (





);
}

 

We can now tint our image with a color like skinColorDef(“#e7b48f”).

But let’s make small improvement.  It’s not obvious what the final color is going to be because the tint color is multiplied by the color in the image.  So let’s make it more intuitive by first looking at the main color in the image (e.g. using the color picker in gimp/photoshop) and then dividing (i.e ‘un-multiplying’) the colorAsString by that color.

For example, the skin color in that girl image is #fff2f2 which is (255,242,228).  So:


divideByImageSkinColor(rgb) {
return {r: rgb.r * (255/255), g: rgb.g * (255/242), b: rgb.b * (255/242)}
}

and modify the skinColorDef like:


skinColorDef(colorAsString) {
const hex = divideByImageSkinColor(hexToRgb(colorAsString));

Now we can just chose the colors directly.  For skin, the Fitzpatrick Scale is a nice place to start:

fitzpatrick-color-chart

We can now use these RGB values directly in our skinColorDef function.  Here’s an example html combobox to select the color: (The onChange function is left to you to implement)



Light
Fair
Medium
Olive
Brown
Black


And that’s it!

Sidenote: Many years ago, I wrote the graphics drivers (when I worked at Imagination Technologies) to accelerate this sort of multiply operation using shaders.  That driver is used in the iPhone, iPad, TomTom, and many other small devices.

Photoshop/gimp layers to SVG

Ever wanted to export multiple layers in a Gimp or Photoshop image, with each layer as its own PNG, but the whole thing then wrapped up as an SVG?

The usefulness is that an artist can create an image of, say, a person, with eyes of various different colours in multiple layers.  Then we can create an SVG file that we can embed in an html page, and then change the color of the eyes through Javascript.

So take this example.  In this image we have a face made up of various layers, and the layers are further grouped in GroupLayers.

So imagine having this image, then in Javascript on your page being able to swap out just the eye image.  Or just the mouth image.

To achieve this, I had to modify an existing gimp python script from 5 years ago that has since bitrotted.  Back when it was written, there was no such thing as group layers, so the script doesn’t work now.  A bit of hacking, and I get:

#!/usr/bin/env python
# -*- coding: <utf-8> -*-
# Author: Erdem Guven <zuencap@yahoo.com>
# Copyright 2016 John Tapsell
# Copyright 2010 Erdem Guven
# Copyright 2009 Chris Mohler
# "Only Visible" and filename formatting introduced by mh
# License: GPL v3+
# Version 0.2
# GIMP plugin to export as SVG

# Save this to ~/.gimp-*/plug-ins/export_svg.py

from gimpfu import *
import os, re

gettext.install("gimp20-python", gimp.locale_directory, unicode=True)

def format_filename(imagename, layer):
	layername = layer.name.decode('utf-8')
	regex = re.compile("[^-\w]", re.UNICODE)
	filename = imagename + '-' + regex.sub('_', layername) + '.png'
	return filename

def export_layers(dupe, layers, imagename, path, only_visible, inkscape_layers):
	images = ""
	for layer in layers:
		if not only_visible or layer.visible:
			style=""
			if layer.opacity != 100.0:
				style="opacity:"+str(layer.opacity/100.0)+";"
			if not layer.visible:
				style+="display:none"
			if style != "":
				style = 'style="'+style+'"'

			if hasattr(layer,"layers"):
				image = '<g inkscape:groupmode="layer" inkscape:label="%s" %s>' % (layer.name.decode('utf-8'),style)
				image += export_layers(dupe, layer.layers, imagename, path, only_visible, inkscape_layers)
				image += '</g>'
				images = image + images
			else:
				filename = format_filename(imagename, layer)
				fullpath = os.path.join(path, filename);
				pdb.file_png_save_defaults(dupe, layer, fullpath, filename)

				image = ""
				if inkscape_layers:
					image = '<g inkscape:groupmode="layer" inkscape:label="%s" %s>' % (layer.name.decode('utf-8'),style)
					style = ""
				image += ('<image xlink:href="%s" x="%d" y="%d" width="%d" height="%d" %s/>\n' %
					(filename,layer.offsets[0],layer.offsets[1],layer.width,layer.height,style))
				if inkscape_layers:
					image += '</g>'
				images = image + images
		dupe.remove_layer(layer)
	return images

def export_as_svg(img, drw, imagename, path, only_visible=False, inkscape_layers=True):
	dupe = img.duplicate()

	images = export_layers(dupe, dupe.layers, imagename, path, only_visible, inkscape_layers)

	svgpath = os.path.join(path, imagename+".svg");
	svgfile = open(svgpath, "w")
	svgfile.write("""<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Generator: GIMP export as svg plugin -->

<svg xmlns:xlink="http://www.w3.org/1999/xlink" """) 	if inkscape_layers: 		svgfile.write('xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" ') 	svgfile.write('width="%d" height="%d">' % (img.width, img.height));
	svgfile.write(images);
	svgfile.write("</svg>");

register(
	proc_name=("python-fu-export-as-svg"),
	blurb=("Export as SVG"),
	help=("Export an svg file and an individual PNG file per layer."),
	author=("Erdem Guven <zuencap@yahoo.com>"),
	copyright=("Erdem Guven"),
	date=("2016"),
	label=("Export as SVG"),
	imagetypes=("*"),
	params=[
		(PF_IMAGE, "img", "Image", None),
		(PF_DRAWABLE, "drw", "Drawable", None),
		(PF_STRING, "imagename", "File prefix for images", "img"),
		(PF_DIRNAME, "path", "Save PNGs here", os.getcwd()),
		(PF_BOOL, "only_visible", "Only Visible Layers?", False),
		(PF_BOOL, "inkscape_layers", "Create Inkscape Layers?", True),
		   ],
	results=[],
	function=(export_as_svg),
	menu=("<Image>/File"),
	domain=("gimp20-python", gimp.locale_directory)
	)

main()

(Note that if you get an error ‘cannot pickle GroupLayers’, this is a bug in gimp. It can be fixed by editing

    350: gimpshelf.shelf[key] = defaults

)

Which when run, produces:

girl-Layer12.png
girl-Layer9.png
girl-Layer11.png
girl-Layer14.png
girl.svg

(I later renamed the layers to something more sensible 🙂 )

The (abbreviated) svg file looks like:


<g inkscape:groupmode="layer" inkscape:label="Expression" >

<g inkscape:groupmode="layer" inkscape:label="Eyes" >

<g inkscape:groupmode="layer" inkscape:label="Layer10" ><image xlink:href="girl-Layer10.png" x="594" y="479" width="311" height="86" /></g>

<g inkscape:groupmode="layer" inkscape:label="Layer14" ><image xlink:href="girl-Layer14.png" x="664" y="470" width="176" height="22" /></g>

<g inkscape:groupmode="layer" inkscape:label="Layer11" ><image xlink:href="girl-Layer11.png" x="614" y="483" width="268" height="85" /></g>

<g inkscape:groupmode="layer" inkscape:label="Layer9" ><image xlink:href="girl-Layer9.png" x="578" y="474" width="339" height="96" /></g>

<g inkscape:groupmode="layer" inkscape:label="Layer12" ><image xlink:href="girl-Layer12.png" x="626" y="514" width="252" height="30" /></g>

</g>

</g>

We can now paste the contents of that SVG directly into our html file, add an id to the groups or image tag, and use CSS or Javascript to set the style to show and hide different layers as needed.

CSS Styling

This all works as-is, but I wanted to go a bit further.  I didn’t actually have different colors of the eyes.  I also wanted to be able to easily change the color.  I use the Inkscape’s Trace Bitmap to turn the layer with the eyes into a vector, like this:

girl-eyecolor

Unfortunately, WordPress.com won’t let me actually use SVG images, so this is a PNG of an SVG created from a PNG….

I used as few colors as possible in the SVG, resulting in just 4 colors used in 4 paths.  I manually edited the SVG, and moved the color style to its own tag, like so:


<defs>
<style type="text/css"><![CDATA[ #eyecolor_darkest { fill:#34435a; } #eyecolor_dark { fill:#5670a1; } #eyecolor_light { fill:#6c8abb; } #eyecolor_lightest { fill:#b4dae5; } ]]></style>

</defs>

<path id="eyecolor_darkest" ..../>

The result is that I now have an svg of a pair of eyes that can be colored through css.  For example, green:

girl-eyecolor_green

Which can now be used directly in the head svg in an html, and styled through normal css:

head_green

Colors

For the sake of completeness, I wanted to let the user change the colors, but not have to make them specify each color individually. I have 4 colors used for the eye, but they are obviously related. Looking at the blue colors in HSL space we get:

RGB:#34435a =  hsl(216, 27%, 28%)
RGB:#5670a1 =  hsl(219, 30%, 48%)
RGB:#6c8abb =  hsl(217, 37%, 58%)
RGB:#b4dae5 =  hsl(193, 49%, 80%)

Annoyingly, the lightest color has a different hue. I viewed this color in gimp, change the hue to 216, then tried to find the closest saturation and value that matched it. 216, 85%, 87% seemed the best fit.

So, armed with this, we now have a way to set the color of the eye with a single hue:

#eyecolor_darkest  =  hsl(hue, 27%, 28%)
#eyecolor_dark     =  hsl(hue, 30%, 48%)
#eyecolor_light    =  hsl(hue, 37%, 58%)
#eyecolor_lightest =  hsl(hue, 85%, 87%)

Or in code:

function setEyeColorHue(hue) {
    document.getElementById("eyecolor_darkest").style.fill = "hsl("+hue+", 27%, 28%)";
    document.getElementById("eyecolor_dark").style.fill = "hsl("+hue+", 30%, 48%)";
    document.getElementById("eyecolor_light").style.fill = "hsl("+hue+", 37%, 58%)";
    document.getElementById("eyecolor_lightest").style.fill = "hsl("+hue+", 85%, 87%)";
}
<label for="hue">Color:</label>
<input type="range" id="hue" min="0" value="216" max="359" step="1" oninput="setEyeColorHue(this.value)" onchange="setEyeColorHue(this.value)"/>

Tinting a more complex image

But what if the image is more complex, and you don’t want to convert it to an SVG?  E.g.

The solution is to apply a filter to multiply the layer by another color.

See my follow up post: Changing the color of image in HTML with an SVG feColorMatrix filter

 

Simple HTML

I frequently want a simple single-file html page that generates some text dynamically based on some inputs at the top of the page.  This can be done in react etc of course, but sometimes my usecase is so simple that it’s an overkill.

For example, to generate some template code based on a few input parameters. Or to make some calculations based on inputs, or to make a customizable story, etc.

With this in mind, I produced the following minimal HTML, using the handlebars processor, that lets me do exactly this:


<!DOCTYPE html>
<html>
<head>
	<meta charset="UTF-8">
    <title>Welcome</title>
    <script type="application/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.0.5/handlebars.min.js"></script>
    <script id="result-template" type="text/x-handlebars-template">
Hello {{name}}!  You are {{age}} years old.  {{#xif "this.age > 18"}} That's really old! {{else}} So young! {{/xif}}


Put the rest of your page here.
    </script>
</head>

<body>
<h2>What is your name?</h2>
Name: <input type="text" id="name" value="Bob"/>

        Age: <input type="number" id="age" value=32 min=0 ><p/><p/>
<div id="resultDiv"></div>
<script>
		var inputs = document.querySelectorAll("input");
		function update() {
			var params = {};
			for (i = 0; i < inputs.length; ++i) {
				params[inputs[i].id] = (inputs[i].type === "number")?Number(inputs[i].value):inputs[i].value;
			}
			document.querySelector("#resultDiv").innerHTML = template(params);
		}
		document.addEventListener("DOMContentLoaded", function() {
			Handlebars.registerHelper("xif", function (expression, options) {
   				 return Handlebars.helpers["x"].apply(this, [expression, options]) ? options.fn(this) : options.inverse(this);
			});
			Handlebars.registerHelper("x", function (expression, options) {
				try { return Function.apply(this, ["window", "return " + expression + " ;"]).call(this, window); } catch (e) { console.warn("{{x " + expression + "}} error: ", e); }
			});

			var source = document.querySelector("#result-template").innerHTML;
			template = Handlebars.compile(source);
			for (i = 0; i < inputs.length; ++i) {
				// Use 'input' to update as the user types, or 'change' on loss of focus
				inputs[i].addEventListener("input", update);
			}
			update();
		});
	</script>
</body>
</html>

Which produces a result like:

html

Single-page HTML that changes the page on user input