Planet GNU

Aggregation of development blogs from the GNU Project

October 24, 2016

denemo @ Savannah

Lonely Cactus

guile-curl v0.4 released

I am pleased to announce an small update of guile-curl, which is a library for fetching files by URL from the internet in the GNU Guile dialect of the Scheme programming language.  It supports many protocols, such as https, ftp, tftp, and, most importantly, gopher.

The webpage for the library is at

The development tree is at

The latest source archive is at

The NEWS for this release is
- support for READDATA callbacks
- bug fixes, especially with respect to garbage collection on newer versions of Guile.

There is no mailing list per se, but, you can contact the project either via or by e-mailing me directly at

Mike Gran

by Mike ( at October 24, 2016 02:01 AM

October 23, 2016

metahtml @ Savannah

Development of a New MetaHTML

MetaHTML is being ported to modern GNU/Linux systems by a small team of eager contributors. We are happy to announce the new developments in the world of GNU MetaHTML.

MetaHTML will eventually be compatible with the latest versions of its dependencies and dependents. We're working on getting it building now, but it's very makeshift. We're hoping to fix that.

Thank you for your interest in the project and I hope you follow development, report bugs, and above all, remain free.

~ Common Era (Ben Sutter), contributor to MetaHTML

by Ben Sutter at October 23, 2016 01:54 AM

October 22, 2016

parallel @ Savannah

GNU Parallel 20161022 ('Matthew') released [stable]

GNU Parallel 20161022 ('Matthew') [stable] has been released. It is available for download at:

No new functionality was introduced so this is a good candidate for a stable release.

Haiku of the month:

Speed is good for you
Multi speed is better still
Use GNU Parallel
-- Ole Tange

New in this release:

  • First command no longer gets stdin as it causes unpredictable issues.
  • Bug fixes and man page updates.

GNU Parallel - For people who live life in the parallel lane.

About GNU Parallel

GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

You can find more about GNU Parallel at:

You can install GNU Parallel in just 10 seconds with: (wget -O - || curl | bash

Watch the intro video on

Walk through the tutorial (man parallel_tutorial). Your commandline will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login: The USENIX Magazine, February 2011:42-47.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ lists
  • Get the merchandise
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference

If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)

If GNU Parallel saves you money:


GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.

About GNU Niceload

GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

by Ole Tange at October 22, 2016 04:07 PM

health @ Savannah

GNU Health 3.0.4 patchset released

Dear community

GNU Health 3.0.4 patchset has been released !

Priority: Medium

Table of Contents

  • About GNU Health Patchsets
  • Updating your system with the GNU Health control Center
  • Installation notes
  • List of issues related to this patchset

About GNU Health Patchsets

We provide "patchsets" to stable releases. Patchsets allow applying bug fixes and updates on production systems. Always try to keep your production system up-to-date with the latest patches.

Patches and Patchsets maximize uptime for production systems, and keep your system updated, without the need to do a whole installation.

For more information about GNU Health patches and patchsets you can visit

NOTE: Patchsets are applied on previously installed systems only. For new, fresh installations, download and install the whole tarball (ie, gnuhealth-3.0.4.tar.gz)

Updating your system with the GNU Health control Center

Starting GNU Health 3.x series, you can do automatic updates on the GNU Health and Tryton kernel and modules using the GNU Health control center program.

Please refer to the administration manual section ( )

The GNU Health control center works on standard installations (those done following the installation manual on wikibooks). Don't use it if you use an alternative method or if your distribution does not follow the GNU Health packaging guidelines.

Summary of this patchset

  • Fix issues on summary reports
  • Fix timezone issue on patient hospitalization admission

Refer to the List of issues related to this patchset for a comprehensive list of fixed bugs.

Installation Notes

You must apply previous patchsets before installing this patchset. If your patchset level is 3.0.3, then just follow the general instructions. You can find the patchsets at GNU Health FTP site (

Follow the general instructions at

List of issues related to this patchset

49405 Error on summary report when no date of birth is assigned to the person
49181 Check admission date with UTC

For detailed information about each issue, you can visit

by Luis Falcon at October 22, 2016 11:33 AM

guile-ncurses @ Savannah

guile-ncurses 2.0 released

I am pleased to announce the release of guile-ncurses 2.0

guile-ncurses is a library for the creation of text user interfaces in the GNU Guile dialect of the Scheme programming language. It is a wrapper to the ncurses TUI library. It contains some basic text display and keyboard and mouse input functionality, as well as a menu widget and a form widget. It also has lower level terminfo and termios functionality.


  • changes to the configuration and build process
    • Compiled scheme files (.go files) are created
    • The new default behavior of the install is to put the scheme and compiled libraries into the directories expected by the current Guile. Previously, the libraries and scheme files would be installed under $prefix (usually /usr/local) unless the --with-guilesitedir configure option was set. A new configure option --with-gnu-filesystem-hierarchy is available if the old default behavior of installing under $prefix is desired.
    • Guile 2.0 is expected. Guile 1.6, 1.8 and 2.1 are not tested.
    • the "wide" UTF-8-capable version of the ncurses libraries is now used by default. The configure option '--without-ncursesw' can be used to explicitly compile with the 8-bit-only "narrow" ncurses libraries.
    • building the termios functionality is now the default. Before it was not built by default. To not build the termios functionality, use the --disable-termios configure option.
  • changes to the (ncurses curses) base library
    • new procedures 'use-extended-names', 'getdelay'
    • new panel iterator functions 'panels-map' and 'panels-for-each'
    • new panel equivalence function 'panel=?'
    • new window function 'resize'
    • new key/mouse constants KEY_EVENT, NCURSES_MOUSE_VERSION, the BUTTON5 constants
    • for systems with no wchar_t conversions availble in libunistring, such as FreeBSD, character conversions assume that wchar_t is UTF-32
  • changes to the (ncurses menu) library
    • new constant O_MOUSE_EVENT
  • changes to the (ncurses form) library
    • new procedure 'current-field'
  • changes to the (ncurses extra) library
    • the termios API is completely re-written, and incompatible with the previous version
  • the test suite has be reworked. tests are more specific and informative. the test-suite.log output file is easier to read.

by Mike Gran at October 22, 2016 05:54 AM

October 21, 2016

unifont @ Savannah

Unifont 9.0.03 Released

Unifont 9.0.03 is released. The main changes are the addition of the Pikto and Tonal ConScript Unicode Registry scripts. The package and related files can be downloaded at

by Paul Hardy at October 21, 2016 06:26 PM

FSF Blogs

October 2016: Docteur Honoris Causa Stallman

Last week, on October 11th, RMS received an honorary doctorate from the Université Pierre et Marie Curie, in Paris, France. In anticipation of the merger between the UPMC and the Université Paris-Sorbonne, which will be finalized in January 2018, the two institutions organized, in the great amphitheater of the Sorbonne, a joint ceremony that recognized both achievers in the sciences and in medicine, as well as ones in the arts and the humanities. The UPMC stated, “This joint ceremony was the occasion to welcome with pride into two communities, soon to be joined as one, these men and women of remarkable achievement who are committed to the service of the common good.” Six women and eight men1 were honored, “each one for the essential role they play in their discipline.”

Dr. Alice P. Gast, president of Imperial College London, in her speech to represent the eight doctors in medicine and in the sciences,2 stated that she and her fellow honorees were “proud to be joining the group of 130 eminent scientists who have received this recognition since 1975.” “We have all had different starting points,” she continued, “and very different paths. But our carriers and our love of science have one thing in common: the ardent will to contribute to the progress of society.”

Some of the honorees, including RMS, along with their respective hosts.
(Photo under CC BY-SA 3.0 and courtesy of Lionel Allorge.)

The UPMC's Dr. Nathalie Drach-Temam introducing RMS.
(Photos under CC BY-SA 3.0 and courtesy of Laurent Ardhuin for the UPMC.)

UPMC president Jean Chambaz, in red, awards RMS the diploma and the medal, while Paris-Sorbonne University president Barthélémy Jobert, in yellow, drapes the three-banded, doctoral epitoge over RMS's left shoulder.
(Photos under CC BY-SA 3.0 and courtesy of Laurent Ardhuin for the UPMC.)

The honorees and their hosts and representatives from the two universities. Those in the burgundy-colored university robes are from the sciences; those in the red, from medicine; those in the yellow, from the humanities.
(Photo under CC BY-SA 3.0 and courtesy of Laurent Ardhuin for the UPMC.)

RMS enjoying a snack and socializing at the post-ceremony reception.
(Photos under CC BY-SA 3.0 and courtesy of Lionel Allorge.)

Later that day, RMS was on the UPMC's Jussieu campus to deliver his speech “What Makes Digital Inclusion Good or Bad?”3 to an audience of about four to five hundred people.

Please fill out our contact form, so that we can inform you about future events in and around Paris. Please see for a full list of all of RMS's confirmed engagements, and contact if you'd like him to come speak.

Thank you to the UPMC for having recognized RMS in this way!

1. You can find a list of all the honorees here.
2. Read Alice P. Gast's full speech here (in French).
3. A recording of RMS's speech will soon be available both here and in our audio-video archive.

October 21, 2016 02:33 PM

FSF Events

Richard Stallman - "Should We Have More Surveillance Than the USSR?" (Lisbon, Portugal)

Digital technology has enabled governments to impose surveillance that Stalin could only dream of, making it next to impossible to talk with a reporter undetected. This puts democracy in danger. Stallman will present the absolute limit on general surveillance in a democracy, and suggest ways to design systems not to collect dossiers on all citizens.

Richard Stallman's speech will be nontechnical, admission is gratis, and the public is encouraged to attend.

Location: (room number to be determined) ISCTE, Av.ª das Forças Armadas

Please fill out our contact form, so that we can contact you about future events in and around Lisbon.

October 21, 2016 02:20 PM

October 20, 2016

FSF Blogs

Friday 'Ghosts' Directory IRC meetup: October 21st starting at a new time! 1pm EDT/17:00 UTC

Participate in supporting the FSD by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on freenode.

Tens of thousands of people visit each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the FSD has been and continues to be a great resource to the world over the past decade, it has the potential of being a resource of even greater value. But it needs your help!

This week we're chasing Ghosts, at a brand new time! Some projects on the directory are currently unmaintained or dormant. While up to date software is always needed, these older projects still present value and the possibility of resurrection. We want to make sure that the resources on the directory are up to date and help people to find both current and older packages. We'll be hunting down these projects this week at a new later time. Lots of our regulars couldn't make it out for our first hour, so we're shifting the schedule back one hour.

If you are eager to help and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly FSD Meetings pages that everyone is welcome to contribute to before, during, and after each meeting.

October 20, 2016 03:08 PM

October 18, 2016

FSF Blogs

Judge Mayer provides a strong case against software patents in Intellectual Ventures v. Symantec

The Free Software Foundation (FSF) has argued for years via its End Software Patents campaign that software patents are a threat to computer users everywhere and need to be abolished. In 2010, the FSF even funded a documentary, "Patent Absurdity: how software patents broke the system," laying out the history of this destructive force. More recently, FSF, the Open Source Initiative, and the Software Freedom Law Center filed an amicus curae in the United State's Supreme Court case of Alice Corp. v. CLS Bank Int’l (Alice). There, we explained the dangers of software patents and argued that "not only do software idea patents fail established tests for patentability; they also violate the First Amendment." It appears that someone on the Federal Circuit (the court that hears appeals on cases involving patents in the U.S.) took note.

That someone is Judge Haldane Robert Mayer, who in a stunning concurrence in Intellectual Ventures v. Symantec (links to a PDF) outlined the case against software patents. The argument will be familiar to those who have read the FSF's Amicus in Alice: software patents fail basic tests for patentability and violate the First Amendment. And while the fact that it is only a concurrence (and not the main opinion of the court) means that it is not settled law, it is a huge step forward in protecting computer users from the dangers of software patents.

Mayer lays out the First Amendment argument against patentability of certain subjects, noting that limits on the subject matter of patents are meant to protect free expression. Under U.S. law, 35 U.S.C § 101 (section 101) lays out the scope of patentable subject matter. In analysing this section, courts have carved out certain subjects as being outside the scope of patentability so as to protect freedom of expression. In particular, abstract ideas and mental process have been found too threatening to the free exchange of ideas to permit them to be locked up in patents. After outlining the basics, Mayer goes on to state that "Most of the First Amendment concerns associated with patent protection could be avoided if this court were willing to acknowledge that Alice sounded the death knell for software patents."

This is a really significant statement, offering clear guidance in the right direction as to how the Supreme Court case should be viewed.

Mayer notes that the Supreme Court in Alice "... explained that the 'mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.' ... Accordingly, '[t]he fact that a computer necessarily exist[s] in the physical, rather than purely conceptual, realm is beside the point'..."

Mayer explains that "Software lies in the antechamber of patentable invention. Because generically-implemented software is an 'idea' insufficiently linked to any defining physical structure other than a standard computer, it is a precursor to technology rather than technology itself."

Mayer continues the line of thought to its practical conclusion: "Given that an 'idea' is not patentable ... and a generic computer is 'beside the point' in the eligibility analysis ... all software implemented on a standard computer should be deemed categorically outside the bounds of section 101."

Mayer even points out that the existence of free software itself is a strong argument against the granting of software patents. That the free software community has flourished for decades without the use of patents "suggests that innovation can flourish in software absent patent" restrictions.

This opinion is exactly the sort of thing we've been fighting for all these years. If only the rest of the court in Intellectual Ventures v. Symantec had joined in with Mayer in recognizing that software should never be subject to patents. While it is a great victory to see this analysis produced by a judge here in the U.S., there's obviously more that needs to be done before the 'death knell' truly sounds for software patents all around the world. Here's what you can do to help bring about that victory even faster:

October 18, 2016 06:08 PM

Free Software Directory meeting recap for October 14th, 2016

Every week free software activists from around the world come together in #fsf on to help improve the Free Software Directory. This recaps the work we accomplished on the Friday, October 14th, 2016 meeting.

This week we were the "Skeleton GNU", with a theme of looking to add substance to entries with only a few details. Unfortunately we were a bit of skeleton crew, with only a few participants. We discussed a bit how to improve turnout and decided to experiment next week with starting the meeting an hour later. Most participants in the past have been joining in the second hour or later of the meeting, so hopefully scheduling the meeting a little later will help overlap better with people's schedules.

The meeting started out working through GNU Project packages to make sure they were robustly detailed on the directory. In the end that turned out to be not a great starting point, as the GNU entries tended to be well documented already, which is a good problem to have.

In working to flesh out entries, we often found that packages with limited information were often quite old and dormant. For some of these the home for the project had disappeared, and new resources were difficult to track down. Iankelling proposed using some new method of tagging these dormant but still potentially useful packages. Even a package that isn't under active development can be useful in some particular cases, and even where a package hasn't been developed in some time, someone can obviously choose to pick it back up and make it live once again. The meeting ended with the decision that handling these 'Ghost' packages would be the theme of next week's meeting.

If you would like to help chase down ghost entries or just help update the directory in general, meet with us every Friday in #fsf on from 1pm to 4pm EDT (17:00 to 20:00 UTC).

October 18, 2016 04:11 PM

October 13, 2016

FSF Blogs

Friday 'Skeleton GNU' Directory IRC meetup: October 14th

Participate in supporting the FSD by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on freenode.

Tens of thousands of people visit each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the FSD has been and continues to be a great resource to the world over the past decade, it has the potential of being a resource of even greater value. But it needs your help!

This week we're the 'Skeleton GNU' looking to beef up bares bones entries with more information. While even the most skeletal entry can be useful, the best ones have robust description, links to documentation and other resources. We'll be looking for entries with minimal information, adding as much as we can and making sure they're up to date.

If you are eager to help and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly FSD Meetings pages that everyone is welcome to contribute to before, during, and after each meeting.

October 13, 2016 07:19 PM

October 2016: photos from Libre Learn Lab, in Cambridge

RMS was in Cambridge, Massachusetts, last week, at the Libre Learn Lab (2016-10-08–09), a summit “for people who create and implement freely licensed resources for education, bringing together educators, policy experts, software developers, hardware hackers, and activists to share best practices and address the challenges of widespread adoption of these resources in education.” He was one of the keynote speakers and, on October 8th, gave his speech “Education for Freedom with Free (Libre) Software,”1 underscoring why exclusive use of free software is essential to educational institutions' role in teaching social responsibility and independence:

(Photos under CC BY-SA 4.0 and courtesy of Rubén Rodriguez.)

Please fill out our contact form, so that we can inform you about future events in and around Cambridge. Please see for a full list of all of RMS's confirmed engagements, and contact if you'd like him to come speak.

Thank you to Mariah Villarreal, Mark Barnett, and Walter Bender for having made this appearance possible!

1. A recording of RMS's speech will soon be available in our audio-video archive.

October 13, 2016 02:07 PM

October 12, 2016

FSF Blogs

Free Software Directory meeting recap for October 7th, 2016

Every week free software activists from around the world come together in #fsf on to help improve the Free Software Directory. This recaps the work we accomplished on the Friday, October 7th, 2016 meeting.

This week was the 'Meeting of the Undead' where we hunted down dead and broken links in the directory. Thanks to the efforts of Iankelling and adfeno, we have a script that runs to find dead external links on the directory and compile them into a list. One common problem we ran into was packages changing the location of their documentation without providing a redirect or updating their directory entry. So, if you are a maintainer out there who is itching to move documentation to a new location, keep in mind that you might be making your documentation harder to find.

We had lots of participants at this week's meeting cracking away at the hordes of undead, but as fmkb pointed out we couldn't get to everything and needed a good way to track entries where a new link couldn't be immediately found. We resolved to simply add notes at the top of the script generated list, but we'll have to revisit and work on a more nuanced solution in the future. Biotim, enrico, and jantwisted were some new faces to the meeting, and bill-auger returned to help mow down dead links.

We didn't end up deciding on a theme for this week during the meeting, but a suggestion afterwards came in that we work as the 'Skeleton GNU' and work on beefing up entries with limited information. While an entry can still be useful with only a short description and a few items of information, robust entries provide better documentation and make the directory an even better resource.

If you would like to help hunt and put meet on the bones of skeleton articles or just help update the directory in general, meet with us every Friday in #fsf on from 12pm to 3pm EDT (16:00 to 19:00 UTC).

October 12, 2016 08:41 PM

Andy Wingo

An incomplete history of language facilities for concurrency

I have lately been in the market for better concurrency facilities in Guile. I want to be able to write network servers and peers that can gracefully, elegantly, and efficiently handle many tens of thousands of clients and other connections, but without blowing the complexity budget. It's a hard nut to crack.

Part of the problem is implementation, but a large part is just figuring out what to do. I have often thought that modern musicians must be crushed under the weight of recorded music history, but it turns out in our humble field that's also the case; there are as many concurrency designs as languages, just about. In this regard, what follows is an incomplete, nuanced, somewhat opinionated history of concurrency facilities in programming languages, with an eye towards what I should "buy" for the Fibers library I have been tinkering on for Guile.

* * *

Modern machines have the raw capability to serve hundreds of thousands of simultaneous long-lived connections, but it’s often hard to manage this at the software level. Fibers tries to solve this problem in a nice way. Before discussing the approach taken in Fibers, it’s worth spending some time on history to see how we got here.

One of the most dominant patterns for concurrency these days is “callbacks”, notably in the Twisted library for Python and the Node.js run-time for JavaScript. The basic observation in the callback approach to concurrency is that the efficient way to handle tens of thousands of connections at once is with low-level operating system facilities like poll or epoll. You add all of the file descriptors that you are interested in to a “poll set” and then ask the operating system which ones are readable or writable, as appropriate. Once the operating system says “yes, file descriptor 7145 is readable”, you can do something with that socket; but what? With callbacks, the answer is “call a user-supplied closure”: a callback, representing the continuation of the computation on that socket.

Building a network service with a callback-oriented concurrency system means breaking the program into little chunks that can run without blocking. Whereever a program could block, instead of just continuing the program, you register a callback. Unfortunately this requirement permeates the program, from top to bottom: you always pay the mental cost of inverting your program’s control flow by turning it into callbacks, and you always incur run-time cost of closure creation, even when the particular I/O could proceed without blocking. It’s a somewhat galling requirement, given that this contortion is required of the programmer, but could be done by the compiler. We Schemers demand better abstractions than manual, obligatory continuation-passing-style conversion.

Callback-based systems also encourage unstructured concurrency, as in practice callbacks are not the only path for data and control flow in a system: usually there is mutable global state as well. Without strong patterns and conventions, callback-based systems often exhibit bugs caused by concurrent reads and writes to global state.

Some of the problems of callbacks can be mitigated by using “promises” or other library-level abstractions; if you’re a Haskell person, you can think of this as lifting all possibly-blocking operations into a monad. If you’re not a Haskeller, that’s cool, neither am I! But if your typey spidey senses are tingling, it’s for good reason: with promises, your whole program has to be transformed to return promises-for-values instead of values anywhere it would block.

An obvious solution to the control-flow problem of callbacks is to use threads. In the most generic sense, a thread is a language feature which denotes an independent computation. Threads are created by other threads, but fork off and run independently instead of returning to their caller. In a system with threads, there is implicitly a scheduler somewhere that multiplexes the threads so that when one suspends, another can run.

In practice, the concept of threads is often conflated with a particular implementation, kernel threads. Kernel threads are very low-level abstractions that are provided by the operating system. The nice thing about kernel threads is that they can use any CPU that is the kernel knows about. That’s an important factor in today’s computing landscape, where Moore’s law seems to be giving us more cores instead of more gigahertz.

However, as a building block for a highly concurrent system, kernel threads have a few important problems.

One is that kernel threads simply aren’t designed to be allocated in huge numbers, and instead are more optimized to run in a one-per-CPU-core fashion. Their memory usage is relatively high for what should be a lightweight abstraction: some 10 kilobytes at least and often some megabytes, in the form of the thread’s stack. There are ongoing efforts to reduce this for some systems but we cannot expect wide deployment in the next 5 years, if ever. Even in the best case, a hundred thousand kernel threads will take at least a gigabyte of memory, which seems a bit excessive for book-keeping overhead.

Kernel threads can be a bit irritating to schedule, too: when one thread suspends, it’s for a reason, and it can be that user-space knows a good next thread that should run. However because kernel threads are scheduled in the kernel, it’s rarely possible for the kernel to make informed decisions. There are some “user-mode scheduling” facilities that are in development for some systems, but again only for some systems.

The other significant problem is that building non-crashy systems on top of kernel threads is hard to do, not to mention “correct” systems. It’s an embarrassing situation. For one thing, the low-level synchronization primitives that are typically provided with kernel threads, mutexes and condition variables, are not composable. Also, as with callback-oriented concurrency, one thread can silently corrupt another via unstructured mutation of shared state. It’s worse with kernel threads, though: a kernel thread can be interrupted at any point, not just at I/O. And though callback-oriented systems can theoretically operate on multiple CPUs at once, in practice they don’t. This restriction is sometimes touted as a benefit by proponents of callback-oriented systems, because in such a system, the callback invocations have a single, sequential order. With multiple CPUs, this is not the case, as multiple threads can run at the same time, in parallel.

Kernel threads can work. The Java virtual machine does at least manage to prevent low-level memory corruption and to do so with high performance, but still, even Java-based systems that aim for maximum concurrency avoid using a thread per connection because threads use too much memory.

In this context it’s no wonder that there’s a third strain of concurrency: shared-nothing message-passing systems like Erlang. Erlang isolates each thread (called processes in the Erlang world), giving each it its own heap and “mailbox”. Processes can spawn other processes, and the concurrency primitive is message-passing. A process that tries receive a message from an empty mailbox will “block”, from its perspective. In the meantime the system will run other processes. Message sends never block, oddly; instead, sending to a process with many messages pending makes it more likely that Erlang will pre-empt the sending process. It’s a strange tradeoff, but it makes sense when you realize that Erlang was designed for network transparency: the same message send/receive interface can be used to send messages to processes on remote machines as well.

No network is truly transparent, however. At the most basic level, the performance of network sends should be much slower than local sends. Whereas a message sent to a remote process has to be written out byte-by-byte over the network, there is no need to copy immutable data within the same address space. The complexity of a remote message send is O(n) in the size of the message, whereas a local immutable send is O(1). This suggests that hiding the different complexities behind one operator is the wrong thing to do. And indeed, given byte read and write operators over sockets, it’s possible to implement remote message send and receive as a process that serializes and parses messages between a channel and a byte sink or source. In this way we get cheap local channels, and network shims are under the programmer’s control. This is the approach that the Go language takes, and is the one we use in Fibers.

Structuring a concurrent program as separate threads that communicate over channels is an old idea that goes back to Tony Hoare’s work on “Communicating Sequential Processes” (CSP). CSP is an elegant tower of mathematical abstraction whose layers form a pattern language for building concurrent systems that you can still reason about. Interestingly, it does so without any concept of time at all, instead representing a thread’s behavior as a trace of instantaneous events. Threads themselves are like functions that unfold over the possible events to produce the actual event trace seen at run-time.

This view of events as instantaneous happenings extends to communication as well. In CSP, one communication between two threads is modelled as an instantaneous event, partitioning the traces of the two threads into “before” and “after” segments.

Practically speaking, this has ramifications in the Go language, which was heavily inspired by CSP. You might think that a channel is just a an asynchronous queue that blocks when writing to a full queue, or when reading from an empty queue. That’s a bit closer to the Erlang conception of how things should work, though as we mentioned, Erlang simply slows down writes to full mailboxes rather than blocking them entirely. However, that’s not what Go and other systems in the CSP family do; sending a message on a channel will block until there is a receiver available, and vice versa. The threads are said to “rendezvous” at the event.

Unbuffered channels have the interesting property that you can select between sending a message on channel a or channel b, and in the end only one message will be sent; nothing happens until there is a receiver ready to take the message. In this way messages are really owned by threads and never by the channels themselves. You can of course add buffering if you like, simply by making a thread that waits on either sends or receives on a channel, and which buffers sends and makes them available to receives. It’s also possible to add explicit support for buffered channels, as Go, core.async, and many other systems do, which can reduce the number of context switches as there is no explicit buffer thread.

Whether to buffer or not to buffer is a tricky choice. It’s possible to implement singly-buffered channels in a system like Erlang via an explicit send/acknowlege protocol, though it seems difficult to implement completely unbuffered channels. As we mentioned, it’s possible to add buffering to an unbuffered system by the introduction of explicit buffer threads. In the end though in Fibers we follow CSP’s lead so that we can implement the nice select behavior that we mentioned above.

As a final point, select is OK but is not a great language abstraction. Say you call a function and it returns some kind of asynchronous result which you then have to select on. It could return this result as a channel, and that would be fine: you can add that channel to the other channels in your select set and you are good. However, what if what the function does is receive a message on a channel, then do something with the message? In that case the function should return a channel, plus a continuation (as a closure or something). If select results in a message being received over that channel, then we call the continuation on the message. Fine. But, what if the function itself wanted to select over some channels? It could return multiple channels and continuations, but that becomes unwieldy.

What we need is an abstraction over asynchronous operations, and that is the main idea of a CSP-derived system called “Concurrent ML” (CML). Originally implemented as a library on top of Standard ML of New Jersey by John Reppy, CML provides this abstraction, which in Fibers is called an operation1. Calling send-operation on a channel returns an operation, which is just a value. Operations are like closures in a way; a closure wraps up code in its environment, which can be later called many times or not at all. Operations likewise can be performed2 many times or not at all; performing an operation is like calling a function. The interesting part is that you can compose operations via the wrap-operation and choice-operation combinators. The former lets you bundle up an operation and a continuation. The latter lets you construct an operation that chooses over a number of operations. Calling perform-operation on a choice operation will perform one and only one of the choices. Performing an operation will call its wrap-operation continuation on the resulting values.

While it’s possible to implement Concurrent ML in terms of Go’s channels and baked-in select statement, it’s more expressive to do it the other way around, as that also lets us implement other operations types besides channel send and receive, for example timeouts and condition variables.

1 CML uses the term event, but I find this to be a confusing name. In this isolated article my terminology probably looks confusing, but in the context of the library I think it can be OK. The jury is out though.

2 In CML, synchronized.

* * *

Well, that's my limited understanding of the crushing weight of history. Note that part of this article is now in the Fibers manual.

Thanks very much to Matthew Flatt, Matthias Felleisen, and Michael Sperber for pushing me towards CML. In the beginning I thought its benefits were small and complication large, but now I see it as being the reverse. Happy hacking :)

by Andy Wingo at October 12, 2016 01:45 PM

guile @ Savannah

GNU Guile 2.0.13 released [security fixes]

We've just released a new version of GNU Guile, version 2.0.13, which is a security release for Guile (see the original announcement).

This handles a significant security vulnerability affecting the live REPL, CVE-2016-8606. Due to the nature of this bug, Guile applications themselves in general aren't vulnerable, but Guile developers are. Arbitrary Scheme code may be used to attack your system in this scenario. (A more minor security issue is also addressed, CVE-2016-8605.)

There is also a lesson here that applies beyond Guile: the presumption that "localhost" is only accessible by local users can't be guaranteed by modern operating system environments. If you are looking to provide local-execution-only, we recommend using Unix domain sockets or named pipes. Don't rely on localhost plus some port.

To give context, Guile supports a nice live-hacking feature where a user can expose a REPL to connect to, through Geiser or so on. This allows Guile users to hack programs even while programs are running.

When using the live hacking feature, the default in Guile has been to expose a port over localhost to which code may be passed. The assumption for this is that only a local user may write to localhost, so it should be safe. Unfortunately, users simultaneously developing Guile and operating modern browsers are vulnerable to a combination of an HTML form protocol attack and a DNS rebinding attack. How to combine these attacks is published in the article How to steal any developer's local database.

In Guile's case, the general idea is that you visit some site which presumably loads some JavaScript code (or tricks the developer into pressing a button which performs a POST), and the site operator switches the DNS from their own IP to Then a POST is done from the website to with the body containing Scheme code. This code is then executed by the Guile interpreter on the listening port.

The version we are releasing mitigates this problem by detecting incoming HTTP connections and closing them before executing any code.

However, there is a better long term solution, which is already available even to users of older versions of Guile: Guile supports Unix domain sockets in POSIX environments. For example, users may run the command:

to open and listen to a socket at /tmp/guile-socket. Geiser users may then connect using M-x geiser-connect-local. This is considerably safer.

We hope that other program authors take heed of this lesson as well: many programs make use of localhost + port as a way of limiting connections. Unfortunately, in today's complex networked environment, this isn't a safe assumption. It's very difficult to predict what programs may provide a way of chaining requests to an application listening on localhost, and certainly difficult on a system where web browsers are involved. Take heed!

(This post originally appeared on the guile-users mailing list.)

by Christopher Allan Webber at October 12, 2016 12:56 PM

October 11, 2016

FSF News

The Free Software Foundation seeks nominations for the 19th annual Free Software Awards

Award for the Advancement of Free Software

This award is presented annually by FSF president Richard Stallman to an individual who has made a great contribution to the progress and development of free software, through activities that accord with the spirit of free software.

Individuals who describe their projects as "open" instead of "free" are eligible nonetheless, provided the software is in fact free/libre.

Last year, Werner Koch was recognized with the Award for the Advancement of Free Software for his work on GnuPG, the de facto tool for encrypted communication. Koch joined a prestigious list of previous winners including Sébastien Jodogne, Matthew Garrett, Dr. Fernando Perez, Yukihiro Matsumoto, Rob Savoye, John Gilmore, Wietse Venema, Harald Welte, Ted Ts'o, Andrew Tridgell, Theo de Raadt, Alan Cox, Larry Lessig, Guido van Rossum, Brian Paul, Miguel de Icaza, and Larry Wall.

Award for Projects of Social Benefit

Nominations are also sought for the 2016 Award for Projects of Social Benefit.

This award is presented to the project or team responsible for applying free software, or the ideas of the free software movement, in a project that intentionally and significantly benefits society in other aspects of life.

The award recognizes projects or teams that encourage people to cooperate in freedom to accomplish tasks of great social benefit, and those that apply free software ideas and lessons outside the free software community. A long-term commitment to one's project (or the potential for a long-term commitment) is crucial to this end.

This award stresses the use of free software in the service of humanity. The FSF has deliberately chosen this broad criterion so that many different areas of activity can be considered. However, one area that is not included is that of free software itself. Projects with a primary goal of promoting or advancing free software are not eligible for this award (the FSF honors individuals working on those projects with its annual Award for the Advancement of Free Software).

The award committee will consider any project or team that uses free software or its philosophy to address a goal important to society. To qualify, a project must use free software, produce free documentation, or use the idea of free software as defined in the Free Software Definition. Projects that promote or depend on the use of non-free software are not eligible. Commercial projects are not excluded, but commercial success is not the metric for judging projects.

Last year, the Library Freedom Project received the award. A partnership among librarians, technologists, attorneys, and privacy advocates which aims to make real the promise of intellectual freedom in libraries, the Library Freedom Project teaches librarians about surveillance threats, privacy rights and responsibilities, and offers digital tools to stop surveillance, all with the aim of creating a privacy-centric paradigm shift in libraries and the local communities they serve. Notably, the project helps libraries launch Tor exit nodes.

Other previous winners have included Reglue, the GNOME Outreach Program for Women (now Outreachy), OpenMRS, GNU Health, Tor, the Internet Archive, Creative Commons, Groklaw, the Sahana project, and Wikipedia.


In the case of both awards, previous winners are not eligible for nomination, but renomination of other previous nominees is encouraged. Only individuals are eligible for nomination for the Advancement of Free Software Award (not projects), and only projects can be nominated for the Social Benefit Award (not individuals). For a list of previous winners, please visit

Current FSF staff and board members, as well as award committee members, are not eligible.

Winners will be decided by a committee to be announced, including several previous winners. Last year's committee was:

  • Suresh Ramasubramanian
  • Rob Savoye
  • Jonas Öberg
  • Fernanda Weiden
  • Wietse Venema
  • Matthew Garrett
  • Vernor Vinge
  • Hong Feng
  • Andrew Tridgell
  • Marina Zhurakhinskaya
  • Richard Stallman


After reviewing the eligibility rules above, please click on the links below to submit your nominations. All nominations need to be submited before Sunday, November 6th, 2016 at 23:59 UTC.

Information about the previous awards can be found at Winners will be announced at an awards ceremony at the LibrePlanet conference, March 25-26 2016, in the Boston area.

About the Free Software Foundation

The Free Software Foundation, founded in 1985, is dedicated to promoting computer users' right to use, study, copy, modify, and redistribute computer programs. The FSF promotes the development and use of free (as in freedom) software -- particularly the GNU operating system and its GNU/Linux variants -- and free documentation for free software. The FSF also helps to spread awareness of the ethical and political issues of freedom in the use of software, and its Web sites, located at and, are an important source of information about GNU/Linux. Donations to support the FSF's work can be made at Its headquarters are in Boston, MA, USA.

More information about the FSF, as well as important information for journalists and publishers, is at

Media Contacts

Georgia Young Program Manager Free Software Foundation +1 (617) 542 5942

October 11, 2016 06:30 PM

FSF Events

John Sullivan - "Freedom Embedded: Devices That Respect Users and Communities" (Raleigh, NC)

John Sullivan will be speaking at All Things Open (2016-10-26–27). His speech will be nontechnical and the public is encouraged to attend.

GNU and Linux are now embedded in more kinds of hardware than ever, but nearly always only by requiring proprietary bits. The world’s most popular tablets and phones are based on a free core system loaded with nonfree software on top. How do we get the freedom we all want, and what is the market for that? The Free Software Foundation has a certification program called “Respects Your Freedom” (RYF) that awards a certification mark to hardware meeting a set of free software standards ( Embedded and IoT devices are a major target for the future of this program.
RYF has already made significant gains, especially over the last few years, certifying USB wifi adapters, 3D printers, home wifi routers, and earlier generation laptops. Even bigger things are planned, and most involve the embedded world. Get updates on what’s in store, learn what it takes to get your product certified, hear about the impact of certification so far and the community that has formed around the program, and discuss possible improvements to the standards.

Location: Raleigh Convention Center, Raleigh, NC

Please fill out our contact form, so that we can contact you about future events in and around Raleigh.

October 11, 2016 11:35 AM

October 10, 2016

FSF Events

Richard Stallman to speak in Alcala de Henares, Spain

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

El título y la hora de la charla serán determinados.

Lugar: por determinar

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Alcalá de Henares.

October 10, 2016 04:45 PM

October 07, 2016

FSF Events

Richard Stallman to speak at GNU Health Con 2016 (Las Palmas, Spain)

Richard Stallman will be speaking at I International GNU Health Conference (2016-11-18–20). His speech will be nontechnical and the public is encouraged to attend.

Speech topic and start time to be determined.

Location: to be determined

Please fill out our contact form, so that we can contact you about future events in and around Las Palmas.

October 07, 2016 03:25 PM

Richard Stallman - "El Software Libre y Tu Libertad" (Barcelona, Spain)

Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

Lugar: Sala de actos Instituto Ausiàs Mach, Avenida de Esplugues, 38 08034 (metro linea 3, Parada Zona Universitaria), Barcelona, Spain

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Barcelona.

El registro para esta charla se puede hacer de modo anónimo. (No es obligatorio, pero se agradecerá para facilitar la organización del evento.)

October 07, 2016 02:45 PM

Richard Stallman - "El software libre en la ética y en la práctica" (Salamanca, Spain)

Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

Lugar: Salón de Actos de la Facultad de Ciencias, Universidad de Salamanca Plaza de los Caídos, S/N, 37008 Salamanca, España

El registro para esta charla se puede hacer de modo anónimo. (No es obligatorio, pero se agradecerá para facilitar la organización del evento.)

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Salamanca.

October 07, 2016 02:35 PM

GNUnet News

Reverse lookups in GNS

DNS allows to resolve the name of an IP address. This is sometimes called "reverse lookup". In fact, it is actually "normal" resolution of a PTR record. The name of such a record would be, for example, The .arpa TLD is managed by IANA.

This blogpost is meant to spread ideas that have been exchanged via private email and might be interesting for a broader audience. If you feel like you have useful comments, don't hesitate to do so.

by Martin Schanzenbach at October 07, 2016 09:16 AM

October 06, 2016

FSF Blogs

Register now for LibrePlanet 2017: "The Roots of Freedom" March 25-26, 2017 in Boston, MA

LibrePlanet 2017
March 25-26
Greater Boston

Gratis for students and FSF associate members

Register now

Clear your calendars for the last weekend of March 2017! Registration for LibrePlanet is now open. The conference will be held March 25-26, 2017, in the Boston area.

Last year, over 350 people participated in LibrePlanet, traveling from 11 countries and 28 U.S. states. And guess what? Students and FSF members attend LibrePlanet gratis. Not a member? Join today to register gratis, or register at our non-member rate.

This year's theme is "The Roots of Freedom." Some talks at this year's conference will look back at the historical "roots" of the free software movement, including the Four Freedoms, the GNU General Public License and copyleft, and the focus on strong security and privacy protections. Others will look at the movement's strong, foundational "roots" -- and the exciting developments that grow from them in education, business, activism, and the arts.

Want to speak at LibrePlanet?

You have just a few weeks left to submit a session proposal -- the deadline is November 16th, 2016 at 18:59 EST (23:59 UTC).

Do you have questions about what makes a strong proposal? We're hosting an information session to answer any questions you have about submitting a proposal for LibrePlanet on Tuesday, November 1st from 15:00 - 18:00 EDT (19:00 - 22:00 UTC). Join us in the #libreplanet IRC channel on Don't use IRC? Email your questions to

Need help traveling to LibrePlanet?

We have a limited amount of funding to bring conference participants to LibrePlanet from all around the world. You can apply for a scholarship starting now -- the application deadline is Monday, December 1st, 2016 at 10:00 EST in the morning (15:00 UTC). Scholarship recipients will be notified in mid-December.

If you don't need a scholarship, you can help us welcome all types of people to LibrePlanet 2016 by making a contribution to the conference's scholarship fund now, or when you register for LibrePlanet.

Will your company sponsor LibrePlanet?

Would your employer consider sponsoring LibrePlanet? We'll be providing the info you need to persuade them soon, but you can also contact us now at for more information.

LibrePlanet is an energizing, friendly event where you'll meet activists, students, developers, and leaders -- and you may get involved in a new project, gain new energy for free software activism, and make new friends. We work hard to ensure a welcoming environment for all, and we have a safe space policy. Don't wait to register: we want you here with us on March 25-26, 2017 for LibrePlanet 2017.

Check out LibrePlanet 2016's video of testimonials about why the free software community loves this annual event!

October 06, 2016 07:05 PM

Christopher Allan Webber

An even more distributed ActivityPub

So ActivityPub is nearing Candidate Recommendation status. If you want to hear a lot more about that whole process of getting there, and my recent trip to TPAC, and more, I wrote a post on the MediaGoblin blog about it.

Last night my brother Stephen came over and he was talking about how he wished ActivityPub was more of a "transactional" system. I've been thinking about this myself. ActivityPub as it is designed is made for the social network of 2014 more or less: trying to reproduce what the silos do, which is mutate a big database for specific objects, but reproduce that in a distributed way. Well, mutating distributed systems is a bit risky. Can we do better, without throwing out the majority of the system? I think it's possible, with a couple of tweaks.

  • The summary is to move to objects and pointers to objects. There's no mutation, only "changing" pointers (and even this is done via appending to a log, mostly).

    If you're familiar with git, you could think of the objects as well, objects, and the pointers as branches.

    Except... the log isn't in the objects pointing at their previous revisions really, the logging is on the pointers:

    [pointer id] => [note content id]
  • There's (activitystreams) objects (which may be content addressed, to be more robust), and then "pointers" to those, via signed pointer-logs.

  • The only mutation in the system is that the "pointers", which are signed logs (substitute "logs" for "ledger" and I guess that makes it a "blockchain" loosely), are append-only structures that say where the new content is. If something changes a lot, it can have "checkpoints". So, you can ignore old stuff eventually.

  • Updating content means making a new object, and updating the pointer-log to point to it.

  • This of course leads to a problem: what identifier should objects use to point at each other? The "content" id, or the "pointer-log" id? One route is that when one object links to another object, it could link to both the pointer-log id and the object id, but that hardly seems desirable...

  • Maybe the best route is to have all content ids point back at their official log id... this isn't as crazy as it sounds! Have a three step process for creating a brand new object:

    • Open a new pointer-log, which is empty, and get the identifier
    • Create the new object with all its content, and also add a link back to the pointer-log in the content's body
    • Add the new object as the first item in the pointer-log
  • At this point, I think we can get rid of all side effects in ActivityPub! The only mutation thing is append-only to that pointer-log. As for everything else:

    • Create just means "This is the first time you've seen this object." And in fact, we could probably drop Create in a system like this, because we don't need it.
    • Update is really just informing that there's a new entry on the pointer-log.
    • Delete... well, you can delete your own copy. You're mostly informing other servers to delete their copy, but they have a choice if they really will... though that's always been true! You now can also switch to the nice property that removing old content is now really garbage collection :)
  • Addressing and distribution still happens in the same, previous ways it did, I assume? So, you still won't get access to an object unless you have permissions? Though that gets more confusing if you use the (optional) content addressed storage here.

  • You now get a whole lot of things for free:

    • You have a built in history log of everything
    • Even if someone else's node goes down, you can keep a copy of all their content, and keep around the signatures to show that yeah, that really was the content they put there!
    • You could theoretically distribute storage pretty nicely
    • Updates/deletes are less dangerous

(Thanks to Steve for encouraging me to think this through more clearly, and lending your own thoughts, a lot of which is represented here! Thanks also to Manu Sporny who was the first to get me thinking along these lines with some comments at TPAC. Though, any mistakes in the design are mine...)

Of course, you can hit even more distributed-system-nerd points by tossing in the possibility of encrypting everything in the system, but let's leave that as an exercise for the reader. (It's not too much extra work if you already have public keys on profiles.)

Anyway, is this likely to happen? Well, time is running out in the group, so I'm unlikely to push for it in this iteration. But the good news, as I said, is that I think it can be built on top without too much extra work... The systems might even be straight-up compatible, and eventually the old mutation-heavy-system could be considered the "crufty" way of doing things.

Architectural astronaut'ing? Maybe! Fun to think about! Hopefully fun to explore. Gotta get the 2014-made-distributed version of the social web out first though. :)

by Christopher Allan Webber at October 06, 2016 09:43 AM

October 05, 2016

GNU MediaGoblin

ActivityPub and MediaGoblin at TPAC 2016 (or: ActivityPub needs your review!)

Screenshot of ActivityPub Working Draft

Hello everyone! We have a lot of news to cover, but I’m going to jump right into the thick of it: we’ve been working hard on a new federation (as well as client to server) standard called ActivityPub (formerly ActivityPump). We’ve made tremendous progress, and I was just recently at a face to face meeting at TPAC, the W3C’s big technical conference.

The good news: ActivityPub is aiming to hit Candidate Recommendation status by October 11th. (That’s less than a week away!) However, in order to enter that stage, we need your review! If you have any interest in the decentralized web, you can help. All you have to do is read the latest editor’s draft and provide feedback. (The earlier the better… maybe a fun weekend project?) You can do this by any of the following:

  • Post issues on the issue tracker (yes, I think it’s ironic and problematic that we’re using GitHub, I picked my battles here, whether right or wrong).
  • Email the social working group comments mailing list.
  • You can email me directly. Email cwebber AT dustycloud DOT org, and include “ActivityPub” in the subject. Note that I will be publishing your comments publicly, probably on the Social Working Group wiki!

If you have feedback, we want to hear it! (From anyone, but especially from those who are interested in implementing ActivityPub!) Thank you!

ActivityPub live

Note, the rest of this post is a bit of a dive into behind the scenes decisions and activity in MediaGoblin and ActivityPub… there’s no need to read this part to submit a review!

I’m kind of traveling through time in all the wrong order in this post… but it’s worth jumping forward a bit to see the results of what we’ve done. Over the last many months there’s been a huge push on ActivityPub as a standard, as we’ll talk about. But in order to succeed, I also needed implementations, both of client to server and server to server. Well, I’m happy to say that those did come together…

Pubstrate in action!

This is Pubstrate, an implementation of ActivityStreams and ActivityPub for GNU Guile. Sorry for the sappy embedded love-note, though what’s nice about this is that it’s the first demo I gave to someone else of the client to server functionality working in Pubstrate. But wait, what was the client?

Soci-El in action!

This is soci-el, an ActivityPub client written in… you guessed it… emacs lisp. You can see the rendering of the user’s outbox here as well as the buffers from which the message was composed.

Pretty nerdy! I don’t expect everyone to be using emacs as an ActivityPub client of course… I hope to see various desktop, web, and mobile clients made available. But, Emacs is the most fun for me, and I was time pressed, so there we are.

Everyone loves seeing screenshots, but maybe not all of this stuff makes sense without context. What does this mean for MediaGoblin?

So what’s been going on?

It seems a recurring meme in MediaGoblin land to say “we’ve been quiet, because we’ve been busy” (or maybe historically on every tech blog ever), but I guess I can’t resist repeating the mantra. It’s true! Though the weight of my focus has been shifted from where I expected it to be. From the last few updates over the last year, you would be right to anticipate that the main thing I would be working on would be merging the federation code Jessica has written and getting 1.0 out the door. That was the plan, and we’re still working towards that, but priorities shifted as we realized the opportunities and time pressures we were under with ActivityPub. After the Social Working Group face to face meeting in June, Jessica and I sat down and talked about what we should do. Jessica had recently started working at Igalia (great people!) and was busy with that and other transitions in her life, so we discussed whether we thought it was most sensible to focus my energy on MediaGoblin or on ActivityPub. It was clear that ActivityPub was shaping into a solid specification, but it was also made clear that the Social Working Group’s charter was running out by the end of 2016. We both think ActivityPub is key to MediaGoblin’s success and didn’t want to see our invested time go to waste, so decided my immediate focus should switch to ActivityPub so it could successfully make it as a standard.

Which isn’t doom and gloom for MediaGoblin! MediaGoblin development has continued… the community is good enough that people have been able to work while I’ve been busy. I’m happy to say we also appointed longtime contributor Boris Bobrov as co-maintainer to help reduce me being a bottleneck. (Thank you Boris, and congrats!) Other contributors have also stepped up to the plate. I’m especially thankful of Ben Sturmfels for hosting MediaGoblin hackathons and being so responsive to community members. (And of course, there are many others to thank, too!)

Anyway, I’m not going anywhere, I’ve just been shifting focus to standards work for a bit… but it’s all for the greater good of MediaGoblin. (Well, and the greater federated social web!) Soon, we’ll be putting the work we’re doing on ActivityPub directly into MediaGoblin. When we merge Jessica’s work on federation, we will also retool it so that the first federated release of MediaGoblin will be blazing the trails with ActivityPub.

Both ActivityPub and I personally got a significant boost by a happy visit from friend and Social Working Group co-conspirator Amy Guy. Amy dropped by for an intense four days of standards hacking and scheming, and came up with several significant ways to restructure and improve the document. With her help, we now have much clearer distinction between the use of ActivityPub as a client to server protocol (think mobile applications and desktop applications connecting to your server) vs the server to server protocol (federation). Both of these are now clearly intertwined in the document, but are distinct enough where they can be used and understood separately if desired. We also more clearly established the connections between ActivityPub and the linked data community by explaining ActivityPub’s relationship with Linked Data Notifications.

Amy has a gift for composing standards language, something I’m still struggling to learn (but hopefully getting better with over time). ActivityPub is much better with her hard work. Thank you Amy!

In addition to the standards side of things, in order to get ActivityPub to the next level, we needed to be able to present real live demonstrations of the standard in action. Hence the work on Pubstrate and soci-el mentioned previously. For most of the months before TPAC, I was working furiously day and night to get things ready to show… And then, it was time to head off, and hope it was good enough…


Live demo image, by aaronpk
TPAC demo room photo by Aaron Parecki

Fortunately, all that hard work paid off. The Social Working Group kicked off TPAC with live open-viewing demonstrations of the various standards we’ve been working on. We got in a really solid set of demos from everyone in the group.

From my end, I managed to demo all the core parts of the ActivityPub spec: I wrote a note in my client (soci-el), fired it off to the server (Pubstrate), where it rendered successfully. Then I explained, well, what if you want someone on another machine to see it? So I had another user on a separate Pubstrate instance, added the user over there to the recipients list on my message, shot the message over to the server, crossed my fingers and hey! The live demo of federation succeeded. (Whew!)

In the actual main Social Working Group meeting, we hammered things forward pretty nicely. As said, ActivityPub was positioned to move ahead towards Candidate Recommendation by the 11th. Again, your feedback is most valuable at this time!

Spying image from the campaign video

Oh, one more thing. On the second day of the main Social Working Group meetings, at Amy’s suggestion (apparently she was impressed when I showed her at her visit), I showed off the MediaGoblin campaign video to the group. Apparently I had never done so previously, so it was really nice to hear the reaction: “Holy cow, this is describing exactly the type of stuff we’re working on in the working group!” Yep, exactly… all that stuff you see illustrated in that video, we’re working on putting into code and standards. The dream lives!

So, all this Social Working Group stuff… things are happening! We’re not just goofing off!

TPAC 2016-09-23 goofoff gif by aaronpk
Super serious TPAC group “photo” by Aaron Parecki

… er, right. Not just goofing off! :)

by Christopher Allan Webber at October 05, 2016 04:45 PM

October 04, 2016

FSF Blogs

Free Software Directory meeting recap for September 30th, 2016

Every week free software activists from around the world come together in #fsf on to help improve the Free Software Directory. This recaps the work we accomplished on the Friday, September 30th, 2016 meeting.

This week we had a special theme of focusing on collaboration related software. This was another big category, but we made some great progress in updating tools for helping people work together. The channel also did some great collaboration in improving the tools we use to work on the directory. adfeno and dachary worked together on the participation guide tools section, improving guides for scripting work in the directory. They made a section in the guide for using pywikibot, a command line, to simplify tasks in the directory.

Iankelling and donaldr3 each started from opposite ends of the project management category and got close to meeting in the middle on updating the pages. jgay helped approve a massive batch of updates to the directory, and mattl was also there to help update entries and provide insight on some of the scripting tasks. The meeting ended up running long after its scheduled end time as everyone wanted to keep working on their projects.

adfeno and Iankelling worked long after the end of even the long-running meeting to put together a resource on broken links in the directory. Using pywikibot they were able to generate a listing of dead external links on the directory. Which means that our first meeting in October will be the 'Meeting of the Undead' where we focus on finding and bring back to life dead links.

If you would like to help hunt for the dead links or just help update the directory in general, meet with us every Friday in #fsf on from 12pm to 3pm EDT (16:00 to 19:00 UTC).

October 04, 2016 03:53 PM

Twenty-two new GNU releases in September

For announcements of most new GNU releases, subscribe to the info-gnu mailing list:

To download: nearly all GNU software is available from, or preferably one of its mirrors from You can use the url to be automatically redirected to a (hopefully) nearby and up-to-date mirror.

This month, we welcome Boris Bobrov as the new GNU MediaGoblin co-maintainer.

A number of GNU packages, as well as the GNU operating system as a whole, are looking for maintainers and other assistance: please see if you'd like to help. The general page on how to help GNU is at

If you have a working or partly working program that you'd like to offer to the GNU project as a GNU package, see

As always, please feel free to write to us at with any GNUish questions or suggestions for future installments.

October 04, 2016 02:00 PM

October 03, 2016

gnuastro @ Savannah

Gnuastro 0.2 released

I am pleased to announce the second release of GNU Astronomy Utilities (Gnuastro).

Gnuastro is a collection of programs and library functions for astronomical data manipulation and analysis. The programs are run directory on the command-line, enabling easy and efficient operationS (combined with other installed programs) in shell scripts or Makefiles. The libraries are also usable in C and C++ programs.

The main changes in this version can be seen in the NEWS file which is also distributed with this announcement in footnote 1 (below). The most notable new features are:

  • Gnuastro library: to use the building blocks of Gnuastro's programs in other C and C++ programs. Python wrappers will also be coming with the next releases. In this release (first library release), the main emphasis was on having a robust library build, management, and documentation infra-structure, rather than the actual functions. A lot of work still remains on improving the functions. Most of the interesting functions are also still locked up in the programs. But with this infra-structure in place, the next releases will greatly improve the quantity and quality of the offered library functions.
  • A new program to read and write FITS binary and ASCII tables called `Table' with the executable name `asttable'. It can also show the column information of a FITS table, or select columns with regular expressions on their title.

Here are the compressed sources: (4.0MB) (2.7MB)

Here are the GPG detached signatures, see footnote 2 (below):

Use a mirror for higher download bandwidth:
(Note that mirrors will need some time to synchronize with the main server)

Here are the MD5 and SHA1 checksums of the tarballs.

175ee211b20d6a667a7186f564b5cc79 gnuastro-0.2.tar.gz
2f68cead777e4a6d38d1bccb7c200362 gnuastro-0.2.tar.lz
703918a2f180e3c448c66fc6e8180f40a15bb460 gnuastro-0.2.tar.gz
fbfd3eccf0fc15f49cb25b30ffa5dc95a74d30fc gnuastro-0.2.tar.lz

Mosè Giordano and Vladimir Markelov contributed to the code in this release. I am also very grateful to Alan Lefor, Yahya Sefidbakht, and Mohammad-reza Khellat for their many patient tests during development along with useful comments and suggestions.

Gnuastro 0.2 tarball was bootstrapped with the following tools:
Texinfo 6.1
Autoconf 2.69
Automake 1.15
Libtool 2.4.6
Help2man 1.47.3
Gnulib v0.1-898-ge63f5eb
Autoconf Archives v2016.09.16-8-g8a6b028
Note that these are not installation dependencies, for those, please see

Mohammad Akhlaghi

Postdoctoral research fellow
CRAL (Lyon Observatory), France.

Footnote 1

Noteworthy changes in release 0.2 (library 0.0.0) (2016-10-03) [stable]

Bug fixes
  • Linker errors on some operating systems have been fixed (bug #48076).
New programs or headers
  • Table: a new utility to read and write FITS binary and ASCII tables. It can also print the column information or select columns using regular expressions (task #13579).
  • Shared libraries and headers are now installed. The libraries can be used in C and C++ programs. This release includes the following headers: `gnuastro.h', `array.h', `box.h', `fits.h', `linkedlist.h', `mesh.h', `polygon.h', `qsort.h', `spatialconvolve.h', `statistics.h', `threads.h', `wcs.h', `txtarray.h' (task #13765).
  • Gnuastro now comes with a script in its top source directory (`tmpfs-config-make') to configure and build it in the tmpfs (on the RAM), for those systems that have it. See the new "Configure and build in RAM" section in the book for more (task #14100).
New features
  • Flat profiles in MakeProfiles can be given a profile specific value. The new `--mforflatpix' option MakeProfile will use the value in the magnitude column as a fixed value for each pixel. This can be very useful in defining a mask, or creating segmentation maps or labeled images (task #14115).
  • MakeProfiles can now use input image as canvas. Instead of specifying the WSC and image size paramteres manually. With the new `--inputascanvas' option, MakeProfiles will get this information (along with blank pixels) from an already existing image (task #14116).
  • Type of output in MakeProfiles and Arithmetic can be specified with the `--type' option.
  • Magnitude error column in MakeCatalog with the `--magnitudeerr' option.
  • Arithmetic now has new conditional (task #13870) and logical operators (task #14153) along with an operator for actions only when conditions are true: `where'. The new `isblank' operator will also enable you to select blank, or masked, pixels (task #14146).
  • The `--noerodequant' in NoiseChisel enables it to detect small and sharper profiles by disabling erosion on pixels above a certain quantile (task #14139).
  • MakeCatalog can also calculate the upper limit magnitude for each given object in the image by randomly positioning the object's footprint over undetected regions of the image (task #14167).
  • The source tarball is now also distributed with Lzip for a much better compression ratio and more robust archival file format.
Changes in behavior
  • The two MakeProfiles options `--setconsttonan', `--setconsttomin' have been removed (see `--mforflatpix' above for their alternative).
  • MakeCatalog makes clump catalog only when asked (when the `WCLUMPS' header exists in the objects HDU). This can be very useful in cases like aperture photometry, when the object labels are not generated by NoiseChisel and so a clump image doesn't exist (task #14122).
  • Default cosmological parameters in CosmiCalculator set to Plank 2015 results: A&A (2016), 594, A13 (arXiv 1502.01589).
  • The `--envseed' option (to read random number generator type and seed from the environment) to MakeProfiles and MakeNoise can also be given in the configuration files.

Footnote 2

Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this (similar for `tar.lz' files):

If that command fails because you don't have the required public key,
then run this command to import it:

and rerun the 'gpg --verify' command.

by Mohammad Akhlaghi at October 03, 2016 01:27 PM