By Evan Miller

July 25, 2017

Want to hear a programming joke?

Perl 6.

Want to know my new favorite programming language?

Perl 6.

Shhhh just clone master and run this:

use v6.d.PREVIEW;

my $channel = Channel.new;

my @ten_tasks = (^10).map: {
    start {
        my $thread = $*THREAD.id;
        await $channel;
        say "HOLY MOLEY SOMEONE STOLE MY THREAD" if $thread != $*THREAD.id;
    }
}

$channel.send("Ring ring") for ^10;
$channel.close;

await @ten_tasks;

Guess what it outputs?

Nothing! Just kidding, you’ll see a lot of distressed messages from tasks that passed out wait for the phone to ring, and woke up wearing someone else’s OS thread.¹

Why is this important? Concurrency is hard and if you want M:N thread multiplexing (i.e. WEB SCALE CODE, where application threads aren’t pinned to pthreads) your options today are precisely Erlang, Go, .NET, and Perl 6.

If you’re feeling confused by Erlang, put off by Go, and indifferent to .NET, take a look at Perl 6. Seriously. Sure, it was in development longer than Boyhood — begun in 2000, two years before the film, and released in 2015, two years after it — but maybe if somebody works on something for 15 years, they might actually be making something good. I’m not saying it’s true in this case (and probably not in the case of Boyhood) but at least it’s possible, right?

It’s funny, fifteen years ago everyone was saying how Perl was shooting itself in the foot with a massive language redesign, because the world was clearly going to switch to Python and Ruby. Well everyone did switch to Python and Ruby, but now everyone’s got the itch to switch again because it turns out Python and Ruby weren’t designed for concurrent code execution. Maybe shooting yourself in the foot isn’t such a bad strategy, if what you need are bionic robo-feet.

I’m not going to review Perl 6 as a language here because it’s a massive spec with a lot of dark corners and weird edge cases, but I do want to share some things I’ve discovered about the language run-time. MoarVM, the Perl 6 virtual machine, is a fantastic piece of technology.

Event loop-aware scheduler? Check. Continuations? You got ’em. What happens when you mix the two? Well, you can spin up hundreds of worker routines and have them wait for messages (from, say, the network) without tying up any OS threads. When the routine gets a message, it continues executing on whatever OS thread happens to be available. Hence all the "HOLY MOLEY" in the opening example. It’s like Go or Erlang with more dollar signs, or Nginx or Node.js without the callback cacciatore.²

MoarVM is written in nice clean C, and full of pleasant surprises if you care to dig. There’s no GIL, so unlike Those Other Languages, Perl 6 bytecode can execute concurrently. The Unicode is great, getting "Straße".uc right and using a clever constant-time algorithm for addressing codepoint-combined graphemes. The object model and garbage collector play nicely with the FFI, so interfacing with C is a cinch. The code base is well-written, well-organized, well-documented, and well-tested; little things, like functions having the right size and variables having good names, make it a pleasure to read.

A quick way to judge a language implementation is by inspecting its string concatenation function. If concat is implemented as a realloc and memcpy, well, the upstairs lights probably aren’t set to full brightness. Behold, in MoarVM strings are broken into strands, and strands can repeat without taking up more memory. So this expression:

"All work and no play makes Jack a dull boy" x 1000

Doesn’t make a thousand copies of the string, or even make a thousand pointers to the same string. It’s stored as a short string and a repetition count, and the VM string functions know how to operate efficiently on strings with repeated substrings. Pretty clever, right? Make a hundred million copies; RAM usage barely budges.³

I won’t try to sell you on Perl 6 here — I still don’t entirely understand the language, to be honest — but having spent a few days poking around its run-time, I will tell you that the virtual machine is a promising piece of systems work. Incidentally MoarVM is a relatively new part of the Perl 6 stack, having supplanted ParrotVM, the thing that was supposed to run All The Scripting Languages; MoarVM has a tighter focus on running just Perl 6, and looks like a much stronger foundation for the future.

If you work on implementing programming languages, take a Sunday afternoon and check out the MoarVM code; I suspect even hardened VM hackers can learn at least something from its scheduler and string handling. If you’re drawing up a list of programming languages to try out, do your future self a favor and put Perl 6 on the menu. I can’t tell you if the language will be to your taste, but I can tell you that the chef was famous once, and the kitchen has all the ingredients it needs to make something really special.

Notes

1. Actually, the program will output nothing if you’re using a release of Perl 6 prior to Rakudo Star 2017.07; a caching bug previously caused $*THREAD.id to report an out-of-date value.

2. For the curious: Go switches contexts on I/O, message passes, and function calls. The Erlang VM switches contexts on I/O, message passes, and every 4,000 VM instructions. MoarVM switches contexts on calls to await, which can include message passes and asynchronous I/O promises. Nginx and Node.js never switch context, which is why all the callbacks are necessary.

3. There’s a hard string-length limit of 2³², so don't go too crazy with this feature. Also, Perl 6’s regex / grammar engine seems to flatten everything before matching, so you don't yet get the full benefit of the compressed representation. If you're looking for a project idea…

You’re reading evanmiller.org, a random collection of math, tech, and musings. If you liked this you might also enjoy:

• A Review of Perl 6 (Raku)
• Elixir RAM and the Template of Doom
• You Can’t Dig Upwards
• Why I’m Betting on Julia

Get new articles as they’re published, via LinkedIn, Twitter, or RSS.

Want to look for statistical patterns in your MySQL, PostgreSQL, or SQLite database? My desktop statistics software Wizard can help you analyze more data in less time and communicate discoveries visually without spending days struggling with pointless command syntax. Check it out!

Wizard
Statistics the Mac way

Back to Evan Miller’s home page – Subscribe to RSS – LinkedIn – Twitter