Reminder: here is the first paper of CoCube

The previous article mentioned:

Dr. Liang created a desktop-level robot named “CoCube” (Collaborative-Cube). CoCube uses MicroBlocks as its programming environment. CoCube has many similar features to toio, but also some capabilities that toio does not have, such as extensible hardware interfaces and a closer integration with MicroBlocks.

Recently, I make a CoCube library for Snap!

Motivation

CoCube can already be programmed in MicroBlocks (thanks to Dr. Liang for the excellent work!). There are two motivations for creating the CoCube library for Snap!:

• Use the libraries of Snap! for CoCube
• Making advanced libraries in a host computer with stronger computing power
  • toio also does it this way
  • As Dr. Liang said, “I did the same thing in the ROS system before: the host computer undertakes most of the computing. And send the final angular velocity or linear velocity instructions to the robot”

Idea

The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be. – Alan Kay

Because Snap! and MicroBlocks are both products of the personal computing spirit, both possess exceptional dynamism, I am confident that the goal(programming MicroBlocks devices in Snap!) can be achieved through message passing. In fact, MicroBlocks IDE collaborates with VM through message passing mechanism.

I like to use message passing to design the interoperability mechanism between systems, as this can minimize the coupling between systems.

There is already a library in Snap! for directly passing messages between Snap! and MicroBlocks

This library provides the basic functionality for message passing. I intend to use this library to create the CoCube library, wanting to achieve:

1. Users can get the status of MicroBlocks devices(CoCube) in Snap! (by reporter blocks)
2. Users can send commands (by command blocks) to MicroBlocks devices(CoCube) in Snap!
3. Users only need to program in Snap! without switching back and forth between Snap! and MicroBlocks.
4. As good extensibility as possible

Regarding point 1: get the status of another system. There are usually two ways to do it:

• push style: A system continuously broadcasts messages carrying status data to another system
  • Firmata is this style
  • Most of Scratch3.0’s hardware libraries are of this style, such as the micro:bit library, refer to the analysis of the communication between scratch3.0 and micro:bit.
• pull style: Query the status, usually using request-response style communication.
  • RPC/HTTP is this style
  • Dynatalk is this style

The MicroBlocks messaging(BLE) library provides a PubSub style message passing mechanism. I intend to build dynatalk-over-ble based on it, and then implement a pull style MicroBlocks Client library based on dynatalk-over-ble (similar to dynatalk-over-postmessage)

In our use case, we only need to implement a unidirectional C/S architecture (Snap! as the client, MicroBlocks as the server). Therefore, the complexity of dynatalk-over-ble is only half that of regular dynatalk. The good news is that the half handled by the computationally weaker MicroBlocks is extremely simple.

Implement

On MicroBlocks side

As mentioned earlier, we want:

Users only need to program in Snap! without switching back and forth between Snap! and MicroBlocks.

Therefore, I hope the code on the MicroBlocks side is completely universal. Users only need to save this universal program to the device and then no longer need to open the MicroBlocks platform, allowing them to program the device entirely in Snap!

To achieve this, we save this universal program to the device:

This program is responsible for the following tasks:

• Interpret the received message and “ground” the semantics of the message to the specific functions of the hardware device. The “grounding” work is implemented using the call block, which is similar to eval in other programming languages and can dynamically run custom blocks and VM primitives.
  • handle_message block is used to handle non-blocking messages
  • handle_blocking_message block is used to handle blocking messages

You can think of this program as a server running on hardware, and Snap! as its client. We used the C/S architecture.

On Snap! side

As mentioned earlier, we want:

1. Users can get the status of MicroBlocks devices(CoCube) in Snap! (by reporter blocks)
2. Users can send commands (by command blocks) to MicroBlocks devices(CoCube) in Snap!

We show how they are implemented separately.

For point 1, get the status of MicroBlocks devices(CoCube) in Snap! (by reporter blocks):

For point 2, send commands (by command blocks) to MicroBlocks devices(CoCube) in Snap!

The message is in pure string form, carrying the method name (custom block name or vm primitives) and parameters, separated by ,. Here is an example: call, <msg-id>, Move Forward, 50

How to use CoCube library?

To use the CoCube library, simply open the Snap!.

Load the CoCube library,

Click open MicroBlocks project block to open the MicroBlocks window. After connecting the device and uploading the program, you can disconnect the device and close the MicroBlocks window.

Then start programming in Snap!.

You can even use any vm primitive:

FAQ

How to add more CoCube library?

Download https://snap.codelab.club/libraries/CoCube2.xml. (Open in browser, save as local file)

then:

• Renamed CoCube2.xml to CoCube3.xml
• Edit CoCube3.xml:
  • Replace all 🚙 to another emoji icon (eg: 🚕) (VSCode has a batch replace function)
  • Replace all 🚕 CoCube2 with 🚕 CoCube3
  • Replace all _CoCube2_ with _CoCube3_
• Drag CoCube3.xml into Snap!.

By analogy, you can add any number of CoCube library!

Usage scenarios

When there are multiple MicroBlocks devices, Snap! is perfect as a dashboard to control them all.

How to debug Snap! and MicroBlocks code simultaneously?

Yes, you can debug both software and hardware side code simultaneously! Everything is liveness!

You can connect the same device simultaneously in Snap! and MicroBlocks , then observe the flow of messages between them:

How to save and load MicroBlocks projects in the MicroBlocks window of Snap!?

Save project

Due to iframe limitations, you cannot use the Save button in the menu to save local files

Alternative solution: click Copy project url to clipboard , then paste it into the browser’s address bar, and then save it.

open project

Due to iframe limitations, you cannot use the Open button in the menu to open local files

Alternative solution: drag the local files directly into the MicroBlocks window.

How to find the vm primitives provided by MicroBlocks?

You can find these primitives in the ubl library file, in the form of . [a:b] , such as [display:mbPlot]

Peter shared some examples on Discord:

the push style version

On MicroBlocks side:

This program is responsible for the following tasks:

1. Continuously push out the status of the device (when started block), it can be extended. The current update rate is 20/second (customizable).
2. Interpret the received message ( handle_message block), grounding the semantics of the message to the specific functions of the hardware device. The “grounding” work is achieved using the call block, which is similar to eval in other programming languages, allowing for the dynamic execution of custom blocks and VM primitives.

Project link (Click to Run)

It is worth noting that the speed of push is much faster than pull. If necessary, they can be combined.

Is it possible to create a similar Python client library?

The design of this article can be easily migrated to Python. By using the microblocks_messaging_library and dynatalk-py libraries, it is easy to achieve the same functionality as the MicroBlocks Client library of Snap!.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# pip install -U cocube
from cocube import CoCube

client = CoCube()
found_devices = client.discover(timeout=3) # timeout=5 in windows
print("found devices:", found_devices)
client.connect("MicroBlocks QCQ", timeout=3)

client.display_character("c")

while True:
    position_x = client.position_x
    print(position_x)

Remember to first load this program into the device (same as Snap!)

For more API examples: notebooks

The source code of the CoCube library

Connect multiple devices

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from cocube import CoCube

client = CoCube()
found_devices = client.discover(timeout=5)
print("found devices:", found_devices)
# connect all discovered devices
devices = [CoCube(i) for i in found_devices]

for index, device in enumerate(devices):
    device.display_character(index)

Parallel Control

1
2
3
4
5
6

from concurrent.futures import ThreadPoolExecutor

# Launching parallel tasks: https://docs.python.org/3/library/concurrent.futures.html
with ThreadPoolExecutor() as executor:
    for client in devices:
        executor.submit(client.scroll_text, "hello")