Planet GNU

Aggregation of development blogs from the GNU Project

February 12, 2016

FSF Blogs

You did it! The fundraiser was a success -- are you ready for what's next?

Once again, the free software community helped put the FSF in a strong position to tackle our list of free software initiatives in 2016, by giving $5 or $10,000, becoming a member for the first time, donating a little bit extra this year, and simply helping spread the word. We've said it before, but we'll say it again: we really can't do this work without your passion and generosity.

two people lighting a candle from another

Here's what you helped us achieve:

  • 4,124 membership contributions (annual and monthly)
  • 2,163 one-time donations -- an increase of 629 over last year's fundraiser
  • An average one-time donation of $97.39

And finally, you smashed our goal of 400 new members:

  • 495 new members

In addition to feeling humbled by your generosity, we get some heartening takeaways from those numbers. More people than ever are supporting the FSF's work defending user freedom - in addition to welcoming more new members than in any previous fundraiser, more people gave, and gave more generously, than in the past. Modest giving made an impact, too: those who gave $30 or less contributed nearly $7,000 more than last year, proving that every little bit really does add up. We always appreciate any new membership, but this year, we witnessed an especially moving commitment from hundreds of people who have made the choice to pledge ongoing financial support to the FSF, bringing our total membership to nearly 4,000. Imagine that: four thousand candles lighting the way.

Your support is our mandate. We will continue our work to serve as a guiding light for the free software movement, and to make new lights to burn brighter. In the coming weeks, look for a report on our findings from the FSF Vision survey, an update to the High Priority Free Software Projects list, and rousing conversation and community building at LibrePlanet, as well as more announcements from the Respects Your Freedom certification program, further action resisting the Digital Millennium Copyright Act's Digital Restrictions Management anti-circumvention provisions, and upgrades to the FSF and GNU Project infrastructure.

Once again, thank you for your contributions to the free software movement, and your support of the FSF. Remember that FSF membership offers many benefits, including gratis admission to our annual LibrePlanet conference. Register today to join us at the Massachusetts Institute of Technology's Stata Center on March 19-20, 2016, where we will present LibrePlanet 2016: Fork the System, in partnership with MIT's Student Information Processing Board.

February 12, 2016 07:50 PM

Encryption: probably better than a box of chocolates

A valentine's day crypto robot

I'd go on a date with that robot.
Would you?

Valentine's day is this Sunday. You may be looking forward to it or you may be dreading it; most of us have been through both. But there's something important that you can do regardless of your relationship status:

Ask someone you like -- romantically or otherwise -- to be your cryptovalentine. If they say yes (yikes, nervous!) use the free program GnuPG to set up private and encrypted communication with them.

If one or both of you is new to GnuPG, we recommend our beginner-friendly Email Self-Defense guide. Setting up encrypted communication is a quick activity you can do together whether you are across the room or across the world. And what better way to show love than by helping someone defend their security, privacy, and freedom? This year, we have a new section with tips for teaching Email Self-Defense, which you may find useful when proposing an encryption date.

Once you're done setting up secure communication, share your love with the world by posting about it, or microblogging with the hashtag #ilovefs. Just make sure not to use proprietary software to post.

Microblog about your cryptovalentine.

This is a fun activity, but it can also make a difference. The right to encrypt is endangered around the world, with governments threatening our security and freedom by demanding legal or technological crippling of encryption. Resist with the power of love -- encrypt with your valentine, and tell the world!

And as we've discussed at length, free software is necessary for privacy online. Because nonfree software's code can't be audited publicly, we can never trust it to be free of back doors inserted by accident or by design. We're thankful to all the hardworking free software developers who give us a fighting chance at digital privacy. It goes without saying, but we do love FS.

For more free software Valentine's day fun, like postcards and an #ilovefs photo gallery, visit the Free Software Foundation Europe Web site.

<3

February 12, 2016 03:50 PM

denemo @ Savannah

Release 2.0.2 is out now.

New Features:
Playback View
Fully Typeset View
Click to Play
Drag to play selection
Shift-Drag to loop
Auto-scrolling
Simple Scrolling
Multiple Scroll Points (tempo changes etc)
Play from full score while scrolling a part
Fully-featured MIDI
Cresc. and Dim.
Articulations (staccato etc)
Grace Notes
Individual Tied Notes in Chords
Sustain Pedal … more
LilyPond Export
One file per part
Output divided into measure per line
Measure Number every 10th measure
All Staffs
All Movements
Staff/Voice Editor Improvements
Navigate to other staffs within editor
Visibility/editing of clef, key, time signatures
Guidance for voices typeset on one staff
Dynamics Lines
Display on single line
Distinctive coloring
Instrument Ranges
Out-of-range notes colored red
Easy one-click setting to chord
Instrument Templates
Add staffs pre-configured for chosen instrument
Name, range and transposition included
Substituting Music
Replace music of one staff with that of another
Use, e.g. to typeset Chord Symbols and Fret Diagrams from one set of chords
Fret Diagrams
Include pre-defined Mandolin and Ukulele
Default is guitar
Tablature
Include example for 18th c Lute
Bug Fixes
Ornament with Accidentals above and/or below fixed

by Richard Shann at February 12, 2016 01:42 PM

German Arias

Liberty Eiffel wrapper for IUP toolkit

Since a couple of months ago I’m working in a Liberty Eiffel wrapper to the IUP toolkit. IUP is a multi-platform toolkit for building graphical user interfaces. This is still under development, but I think the current state is enough to start playing with it. Here some screen shots:

iup1

A small converter for inches and centimeters:

iup2

Currently you can use widgets like Windows (in IUP are called Dialogs), buttons, labels, radio buttons, text box, text views (called multi line), sliders, links, combo box, menus and a variety of containers. For the moment there are only four examples, but I will add more later. The example4.e show how create menus and use mnemonics and short cuts. Here some screen shots of this example:

iup4

iup5

To give you an idea about how is the use of this wrapper here an extract of the converter example shown previously:

iup-code

The project is hosted at notabug.org. So you can download it and start playing. Of course you should install first Liberty Eiffel and IUP.


by Germán Arias at February 12, 2016 05:12 AM

February 10, 2016

www @ Savannah

FSF Events

Richard Stallman to speak in Utrecht, Netherlands

Richard Stallman's speech will be nontechnical, admission is gratis, and the public is encouraged to attend.

Speech topic, start time, and exact location to be determined.

Location: TBD

Please fill out our contact form, so that we can contact you about future events in and around Utrecht.

February 10, 2016 12:39 PM

February 09, 2016

FSF Blogs

Confused by license compatibility? A new article by Richard Stallman may help

Richard Stallman has published a new guide on gnu.org titled License compatibility and relicensing. Gnu.org is home to a whole host of resources on free software licensing, including frequently asked questions about GNU licenses and our list of free software licenses. Our license list contains information on which licenses are compatible with the GNU General Public License as well as a brief description of what it means to be compatible. This latest article by Stallman provides a more in–depth explanation of what compatibility means and the different ways in which it is achieved.

As Stallman states, "In general we say that several licenses are compatible if there is a way to merge code under those various licenses while complying with all of them." Even where two licenses are crafted in a such a way that they cannot naturally be combined, however, explicit relicensing provisions can enable code combinations that otherwise wouldn't be permitted. The guide covers all of this and more in depth and clarity.

Resources like this guide are made possible by donations from our community and help from our volunteers. Here is what you can do to help make sure everyone has quality tools for understanding and using free licenses:

February 09, 2016 06:00 PM

www @ Savannah

FSF Events

Richard Stallman to speak in Santa Coloma de Gramenet, Spain

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

El título de la charla será determinado.

Lugar: Institut Puig Castellar, Avinguda Anselm de Riu 10 Santa Coloma de Gramenet (metro 'Can Zam' (L9), 'Santa Coloma' (L1))

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Barcelona.

February 09, 2016 11:53 AM

February 08, 2016

FSF Events

Richard Stallman to speak in Barcelona, Spain

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

El título y la hora de la charla serán determinados.

Lugar: Será determinado.

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Barcelona.

February 08, 2016 02:14 PM

Andy Wingo

a lambda is not (necessarily) a closure

preface

Greets, folks! Check it out: Guile had a whole track devoted to it at FOSDEM this year. OK, so it was only half a day, but there were like a dozen talks! And the room was full all the morning! And -- get this -- I had nothing to do with its organization! I think we can credit the Guix project with the recent surge of interest in Guile; fully half the talks were from people excited about using Guix to solve their problems. Thanks very, very much to Pjotr Prins for organizing the lovely event.

I gave a talk on how the Guile 2.2 compiler and virtual machine could change the way people program. Happily, the video recording came out OK! Video below (or here if that doesn't work), and slides here.

The time was super-limited though and I wasn't able to go into the detail that I'd like. So, dear readers, here we are, with a deeper look on lambda representation in Guile.

a lambda is not (necessarily) a closure

What is this?

(lambda (a b) (+ a b))

If you answer, "it's a lambda expression", you're right! You're also right if you say it's a function -- I mean, lambda makes a function, right? There are lots of things that you could say that would be right, including silly things like "twenty-two characters set in an awkward typeface".

But if you said "it's a closure" -- well you're right in general I guess, like on a semantic what-does-it-mean level, but as far as how Guile represents this thing at run-time, hoo boy are there a number of possibilities, and a closure is just one of them. This article dives into the possibilities, with the goal being to help you update your mental model of "how much do things cost".

In Guile, a lambda expression can be one of the following things at run-time:

  1. Gone

  2. Inlined

  3. Contified

  4. Code pointer

  5. Closure

Let's look into these one-by-one.

lambda: gone

If Guile can prove that a lambda expression is never reached, it won't be present at run-time. The main way this happens is via partial evaluation, but later passes can do this too. In the most basic example, consider the lambda bound to f by this let expression.

(let ((f (lambda ()
           (launch-the-missiles!))))
  42)

Guile has an ,optimize command that can be run at the REPL to show the effect of partial evaluation on your code. These days it's a bit out of date in a way -- it can't show what CPS-based optimization will do to your code -- but for our purposes here it will transform the expression to the following code:

(let ((f (lambda ()
           (launch-the-missiles!))))
  42)
=> 42

So the lambda is gone, big whoop. The interesting thing though is that this happens concurrently with other things that partial evaluation does, so the lambda goes away in this expression too:

(let ((launch? #f)
      (f (lambda ()
           (launch-the-missiles!))))
  (if launch? (f) 'just-kidding))
=> 'just-kidding

lambda: inlined

The other trick that partial evaluation can do with lambda expressions is inlining. Re-taking the example above, if we change launch? to #t, the branch folds the other way and the application (f) inlines:

(let ((launch? #t)
      (f (lambda ()
           (launch-the-missiles!))))
  (if launch? (f) 'just-kidding))
=> (let ((launch? #t)
         (f (lambda ()
              (launch-the-missiles!))))
     (if #t (f) 'just-kidding))
=> (let ((launch? #t)
         (f (lambda ()
              (launch-the-missiles!))))
     (f))
=> (let ((launch? #t)
         (f (lambda ()
              (launch-the-missiles!))))
     ((lambda () (launch-the-missiles!))))
=> (let ((launch? #t)
         (f (lambda ()
              (launch-the-missiles!))))
     (launch-the-missiles!))
=> (launch-the-missiles!)

Here again the lambda is gone, but not because it was unreachable, but because it was inlined into its use. I showed some intermediate steps as well, just so you get a feel about how partial evaluation works. The inlining step is illustrated by the fourth transformation, where the lambda application went away, replaced by its body.

Partial evaluation can also unroll many kinds of recursion:

(letrec ((lp (lambda (n)
               (if (zero? n)
                   n
                   (+ n (lp (1- n)))))))
  (lp 5))
=> 15

The partial evaluator in Guile 2.2 is more or less unchanged from the one in Guile 2.0, so you get these benefits on old Guile as well. Building a good intuition as to what the partial evaluator will do is important if you want to get the best performance out of Guile. Use the ,optimize command at the REPL to see the effects of partial evaluation on any given expression.

lambda: contified

So, here we step into the unknown, in the sense that from here on out, these optimizations are new in Guile 2.2. Unfortunately, they can be hard to see as they aren't really representable in terms of source-to-source transformations over Scheme programs. Consider this program:

(define (count-down n)
  (define loop
    (lambda (n out)
      (let ((out (cons n out)))
        (if (zero? n)
            out
            (loop (1- n) out)))))
  (loop n '()))

It's a little loop that builds a list of integers. The lambda in this loop, bound to loop, will be contified into the body of count-down.

To see that this is the case, we have to use a new tool, ,disassemble (abbreviated ,x). This takes a procedure and prints its bytecode. It can be hard to understand, so I'm going to just point out some "shapes" of disassembly that you can recognize.

> ,x count-down
Disassembly of #<procedure count-down (n)> at #x9775a8:

[...]
L1:
  10    (cons 2 1 2)
  11    (br-if-u64-=-scm 0 1 #f 5) ;; -> L2
  14    (sub/immediate 1 1 1)
  15    (br -5)                    ;; -> L1
L2:
[...]

I've snipped the disassembly to the interesting part. The first thing to notice is that there's just one procedure here: only one time that ,x prints "Disassembly of ...". That means that the lambda was eliminated somehow, either because it was dead or inlined, as described above, or because it was contified. It wasn't dead; we can see that from looking at the ,optimize output, which doesn't significantly change the term. It wasn't inlined either; again, ,optimize can show you this, but consider that because partial evaluation can't determine when the loop would terminate, it won't find a point at which it can stop unrolling the loop. (In practice what happens though is that it tries, hits an effort or code growth limit, then aborts the inlining attempt.)

However, what we see in the disassembly is the body of the loop: we cons something onto a list (the cons), check if a two numbers are equal (br-if-u64-=-scm), and if they are we jump out of the loop (L2). Otherwise we subtract 1 from a number (sub/immediate) and loop (br to L1). That is the loop. So what happened?

Well, if inlining is copying, then contification is rewiring. Guile's compiler was able to see that although it couldn't inline the loop function, it could see all of loop's callers, and that loop always returned to the same "place". (Another way to say this is that loop is always called with the same continuation.) The compiler was then able to incorporate the body of loop into count-down, rewiring calls to loop to continue to loop's beginning, and rewriting returns from loop to proceed to the continuation of the loop call.

a digression on language

These words like "contification" and "continuation" might be unfamiliar to you, and I sympathize. If you know of a better explanation of contification, I welcome any links you might have. The name itself comes from a particular formulation of the intermediate language used in Guile, the so-called "CPS" language. In this language, you convert a program to make it so it never returns: instead, each sub-expression passes its values to its continuation via a tail call. Each continuation is expressed as a lambda expression. See this article for an intro to CPS and how it relates to things you might know like SSA.

Transforming a program into CPS explodes it into a bunch of little lambdas: every subexpression gets its own. You would think this would be a step backwards, if your goal is to eliminate closures in some way. However it's possible to syntactically distinguish between lambda expressions which are only ever used as continuations and those that are used as values. Let's call the former kind of lambda a cont and the latter a function. A cont-lambda can be represented at run-time as a label -- indeed, the disassembly above shows this. It turns out that all lambda expressions introduced by the CPS transformation are conts. Conts form a first-order flow graph, and are basically the same as SSA basic blocks. If you're interested in this kind of thing, see Andrew Kennedy's great paper, Compiling with Continuations, Continued, and see also CPS soup for more on how this evolved in Guile 2.2.

I say all this to give you a vocabulary. Functions that are present in the source program start life as being treated as function-lambdas. Contification takes function-lambda values and turns then into cont-lambda labels, if it can. That's where the name "contification" comes from. For more on contification, see MLton's page on its contification pass, linking to the original paper that introduces the concept.

and we're back

Contification incorporates the body of a function into the flow graph of its caller. Unlike inlining, contification is always an optimization: it never causes code growth, and it enables other optimizations by exposing first-order control flow. (It's easier for the compiler to reason about first-order loops than it is to reason about control flow between higher-order functions.)

Contification is a reliable optimization. If a function's callers are always visible to the compiler, and the function is always called with the same continuation, it will be contified. These are two fairly simple conditions that you can cultivate your instincts to detect and construct.

Contification can also apply to mutually recursive functions, if as a group they are all always called with the same continuation. It's also an iterative process, in the sense that contifying one set of functions can expose enough first-order control flow that more contification opportunities become apparent.

It can take a while to get a feel for when this optimization applies. You have to have a feel for what a continuation is, and what it means for a function's callers to all be visible to the compiler. However, once you do internalize these conditions, contification is something you can expect Guile's compiler to do to your code.

lambda: code pointer

The next representation a lambda might have at run-time is as a code pointer. In this case, the function fails the conditions for contification, but we still avoid allocating a closure.

Here's a little example to illustrate the case.

(define (thing)
  (define (log what)
    (format #t "Very important log message: ~a\n" what))
  (log "ohai")
  (log "kittens")
  (log "donkeys"))

In this example, log is called with three different continuations, so it's not eligible for contification. Unfortunately, this example won't illustrate anything for us because the log function is so small that partial evaluation will succeed in inlining it. (You could determine this for yourself by using ,optimize.) So let's make it bigger, to fool the inliner:

(define (thing)
  (define (log what)
    (format #t "Very important log message: ~a\n" what)
    ;; If `log' is too short, it will be inlined.  Make it bigger.
    (format #t "Did I ever tell you about my chickens\n")
    (format #t "I was going to name one Donkey\n")
    (format #t "I always wanted a donkey\n")
    (format #t "In the end we called her Raveonette\n")
    (format #t "Donkey is not a great name for a chicken\n")
    (newline) (newline) (newline) (newline) (newline))
  (log "ohai")
  (log "kittens")
  (log "donkeys"))

Now if we disassembly it, we do get disassembly for two different functions:

,x thing
Disassembly of #<procedure thing ()> at #x97d704:
[...]

Disassembly of log at #x97d754:
[...]

So, good. We defeated the inliner. Let's look closer at the disassembly of the outer function.

,x thing
Disassembly of #<procedure thing ()> at #x97d704:
[...]
  12    (call-label 3 2 8)              ;; log at #x97d754

Here we see that instead of the generic call instruction, we have the specific call-label instruction which calls a procedure whose code is at a known offset from the calling function.

call-label is indeed a cheaper call than the full call instruction that has to check that the callee is actually a function and so on. But that's not the real optimization here. If all callers of a function are known -- and by this time, you're starting to catch the pattern, I think -- if all callers are known, then the procedure does not need to exist as a value at run-time.

This affords a number of optimization opportunities. Theoretically there are many -- all call sites can be specialized to the specific callee. The callee can have an optimized calling convention that doesn't have anything to do with the generic convention. Effect analysis can understand the side effects and dependencies of the callee in a more precise way. The compiler can consider unboxing some arguments and return values, if it finds that useful.

In Guile though, there's only one real optimization that we do, and that is related to free variables. Currently in Guile, all procedures have a uniform calling convention, in which the procedure being called (the callee) is itself passed as the zeroeth argument, and then the arguments follow on the stack. The function being called accesses its free variables through that zeroeth argument. If however there is no need for the procedure to be represented as a value, we are free to specialize that zeroeth argument.

So, consider a well-known procedure like log above. (By "well-known", we mean that all of log's callers are known.) Since log doesn't actually have any lexically bound free variables, we can just pass in anything as argument zero when invoking it. In practice we pass #f, because it happens to be an easy value to make.

(Why isn't format treated as a free variable in log? Because there is special support from the linker for lazily initializing the locations of variables imported from other modules or defined at the top level instead of within a lexical contour. In short: only variables that are (a) used within the lambda and (b) defined within a let or similar count towards a lambda's free variables.)

For a well-known procedure with only one free variable, we can pass in that free variable as the zeroeth argument. Internally to the function, we rewrite references to that free variable to reference argument 0 instead. This is a neat hack because we can have a lambda with a free variable but which results in no allocation at run-time.

Likewise if there are two free variables -- and this is starting to sound like Alice's restaurant, isn't it -- well we do have to pass in their values to the procedure, but we don't have to build an actual closure object with a tag and a code pointer and all. Pairs happen to be small and have some fast paths in Guile, so we use that. References to the free variables get internally rewritten to be car or cdr of argument 0.

For three or more free variables, we do the same, but with a vector.

For a final trick, a set of mutually recursive procedures whose callers are all known can share the object that collects their free variables. We collect the union of the free variables of all of the procedures, and pack them into a specialized representation as above.

Note that for well-known procedures, all variables that are free in the lambda are also free in the caller; that's why the 1-free-variable substitution works. The lambda is bound in a scope that dominates its callers, but its free variables dominate the lambda so they dominate the callers too. For that reason in this case we could choose to do lambda lifting instead, with no penalty: instead of bundling up the free variables in a heap object, we could pass them as arguments. Dybvig claims this is not a great idea because it increases register pressure. That could be true, but I haven't seen the numbers. Anyway, we do the flat closure thing, so we pack the free vars into data.

All these ideas came pretty much straight from the great Optimizing Closures in O(0) Time by Andrew Keep et al.

lambda: closure

OK! So you have a lambda whose callees are not all visible to the compiler. You need to reify the procedure as a value. That reified procedure-as-value is a closure: an object with a tag, a code pointer, and an array of free variables.

Of course, if the procedure has no free variables, you just have the tag and the code pointer... and because Scheme is semantically squirrely when it comes to the result of (eqv? (lambda () 10) (lambda () 10)) (it's unspecified: lambda expressions don't have identity), we can statically allocate the closure in the binary, as a constant.

Otherwise we do allocate the heap object.

Note however that if a group of mutually recursive procedures has just one entry that is not "well-known", then that procedure clique can share one closure object.

lambda: it's complicated

In summary, a lambda is an abstraction that has many concrete representations. Guile will choose the cheapest representation that it can. If you need to eke out even more performance from your program, having a good mental model of how the abstract maps to the concrete will help you know where to focus your efforts, and what changes might be helpful. Good luck, and happy hacking!

by Andy Wingo at February 08, 2016 10:12 AM

February 06, 2016

texinfo @ Savannah

Texinfo 6.1 released

We have released version 6.1 of Texinfo, the GNU documentation format.

It's available via a mirror (xz is much smaller than gz, but gz is available too just in case):
http://ftpmirror.gnu.org/texinfo/texinfo-6.1.tar.xz
http://ftpmirror.gnu.org/texinfo/texinfo-6.1.tar.gz

Please send any comments to bug-texinfo@gnu.org.

Full announcement:
http://lists.gnu.org/archive/html/bug-texinfo/2016-02/msg00035.html

by Gavin D. Smith at February 06, 2016 01:23 PM

February 05, 2016

grep @ Savannah

grep-2.23 released [stable]

by Jim Meyering at February 05, 2016 12:30 AM

February 04, 2016

gnuzilla @ Savannah

IceCat 38.6.0 release

GNUzilla is the GNU version of the Mozilla suite, and GNU IceCat is the GNU version of the Firefox browser. Its main advantage is an ethical one: it is entirely free software. While the Firefox source code from the Mozilla project is free software, they distribute and recommend non-free software as plug-ins and addons. Also their trademark license restricts distribution in several ways incompatible with freedom 0.
https://www.gnu.org/software/gnuzilla/

The user manual pages are at http://libreplanet.org/wiki/Group:IceCat/
You can contribute by joining the wiki and editing the manuals.

Source tarballs, binaries for generic GNU/Linux systems and translations are available at http://ftp.gnu.org/gnu/gnuzilla/38.6.0/
GPG key ID:D7E04784 GNU IceCat releases
Fingerprint: A573 69A8 BABC 2542 B5A0 368C 3C76 EED7 D7E0 4784
https://savannah.gnu.org/project/memberlist-gpgkeys.php?group=gnuzilla

Changes since v38.5.0

  • Updated to v38.6.0ESR
  • HTTPS-Everywhere updated to 5.1.3
  • HTML5 Video Everywhere updated to 0.3.4
  • Updated to v38.5.2ESR
  • Disabled direct2d acceleration (fix for Windows crashes)

by Ruben Rodriguez at February 04, 2016 11:47 PM

Andy Wingo

guile compiler tasks

Hey! We released Guile 2.1.2, including the unboxing work, and we fixed the slow bootstrap problem by shipping pre-built bootstraps in tarballs. A pretty OK solution in my opinion; check it out!

future work

At this point I think I'm happy with Guile's compiler and VM, enough for now. There is a lot more work to do but it's a good point at which to release a stable series. There will probably be a number of additional pre-releases, but not any more significant compiler/VM work that must be done before a release.

However, I was talking with Guilers at FOSDEM last weekend and we realized that although we do a pretty good job at communicating the haps in compiler-land, we don't do a good job at sharing a roadmap or making it possible for other folks to join the hack. And indeed, it's been difficult to do so while things were changing so much: I had to get things right in my head before joining in the confusion of other people's heads.

In that spirit I'd like to share a list of improvements that it would be nice to make at some point. If you take one of these tasks, be my guest: find me on IRC (wingo on freenode) and let me know, and I'll help as I am able. You need to be somewhat independent; I'm not offering a proper mentoring or anything, more like office hours or something, where you come with the problem you are having and I commiserate and give context/background/advice as I am able.

So with that out of the way, here's a huge list of stuff! Following this, more details on each one.

  1. stripping binaries

  2. full source in binaries

  3. cps in in binaries

  4. linking multiple modules together

  5. linking a single executable

  6. instruction explosion

  7. elisp optimizations

  8. prompt removal

  9. basic register allocation

  10. optimal register allocation

  11. unboxed record fields

  12. textual CPS

  13. avoiding arity checks

  14. unboxed calls and returns

  15. module-level inlining

  16. cross-module inlining

As a bonus, in the end I'll give some notes on native compilation. But first, the hacks!

stripping binaries

Guile uses ELF as its object file format, and currently includes source location information as DWARF data. On space-constrained devices this might be too much. Your task: add a hack to the linker that can strip existing binaries. Read Ian Lance Taylor's linker articles for more background, if you don't know things about linkers yet.

full source in binaries

Wouldn't it be nice if the ELF files that Guile generates actually included the source as well as the line numbers? We could do that, in a separate strippable ELF section. This point is like the reverse of the previous point :)

cps in in binaries

We could also include the CPS IR in ELF files too. This would enable some kinds of link-time optimization and cross-module inlining. You'd need to define a binary format for CPS, like LLVM bitcode or so. Neat stuff :)

linking multiple modules together

Currently in Guile, just about every module is a separate .go file. Loading a module will cause a few stat calls and some seeks and reads and all that. Wouldn't it be nice if you could link together all the .go files that were commonly used into one object? Again this is a linker hack, but it needs support from the run-time as well: when the run-time goes to load a file, it should first check in a registry if that file has been logically provided by some other file. We'd be able to de-duplicate constant data from various modules. However there is an initialization phase when loading a .go file which effectively performs all the relocations needed by constants that need a fix-up at load-time; see the ELF article I linked to above for more. For some uses, it would be OK to produce one relocation/initialization procedure. For others, if you expected to only load a fraction of the modules in a .go file, it would be a lose on startup time,
so you would probably need to support lazy relocation when a module is first loaded.

Anyway, your task would be to write a linker hack that loads a bunch of .go files, finds the relocations in them, de-duplicates the constants, and writes out a combined .go file that includes a table of files contained in it. Good luck :) This hack would work great for Emacs, where it's effectively a form of unexec that doesn't actually rely on unexec.

linking a single executable

In the previous task, you could end up with the small guile binary that links to libguile (or your binary linking to libguile), and then a .go file containing all the modules you are interestd in. It sure would be nice to be able to link those together into just one binary, or at least to link the .go into the Guile binary. If the Guile is statically linked itself, you would have a statically linked application. If it's dynamically linked, it would remain dynamically linked. Again, a linker hack, but one that could provide a nicer way to distribute Guile binaries.

instruction explosion

Now we get more to the compiler side of things. Currently in Guile's VM there are instructions like vector-ref. This is a little silly: there are also instructions to branch on the type of an object (br-if-tc7 in this case), to get the vector's length, and to do a branching integer comparison. Really we should replace vector-ref with a combination of these test-and-branches, with real control flow in the function, and then the actual ref should use some more primitive unchecked memory reference instruction. Optimization could end up hoisting everything but the primitive unchecked memory reference, while preserving safety, which would be a win. But probably in most cases optimization wouldn't manage to do
this, which would be a lose overall because you have more instruction dispatch.

Well, this transformation is something we need for native compilation anyway. I would accept a patch to do this kind of transformation on the master branch, after version 2.2.0 has forked. In theory this would remove most all high level instructions from the VM, making the bytecode closer to a virtual CPU, and likewise making it easier for the compiler to emit native code as it's working at a lower level.

elisp optimizations

Guile implements Emacs Lisp, and does so well. However it hasn't been the focus of a lot of optimization. Emacs has a lot of stuff going on on its side, and so have we, so we haven't managed to replace the Elisp interpreter in Emacs with one written in Guile, though Robin Templeton has brought us a long way forward. We need someone to do both the integration work but also to poke the compiler and make sure it's a clear win.

prompt removal

It's pretty natural to use delimited continuations when compiling some kind of construct that includes a break statement to Guile, whether that compiler is part of Elisp or just implemented as a Scheme macro. But, many instances of prompts can be contified, resulting in no overhead at run-time. Read up on contification and contify the hell out of some prompts!

basic register allocation

Guile usually tries its best to be safe-for-space: only the data which might be used in the future of a program is kept alive, and the rest is available for garbage collection. Notably, this applies to function arguments, temporaries, and lexical variables: if a value is dead, the GC can collect it and re-use its space. However this isn't always what you want. Sometimes you might want to have all variables that are in scope to be available, for better debugging. Your task would be to implement a "slot allocator" (which is really register allocation) that keeps values alive in the parts of the programs that they dominate.

optimal register allocation

On the other hand, our slot allocator -- which is basically register allocation, but for stack slots -- isn't so great. It does OK but you can often end up shuffling values in a loop, which is the worst. Your task would be to implement a proper register allocator: puzzle-solving, graph-coloring, iterative coalescing, something that really tries to do a good job. Good luck!

unboxed record fields

Guile's "structs", on which records are implemented, support unboxed values, but these values are untyped, not really integrated with the record layer, and always boxed in the VM. Your task would be to design a language facility that allows us to declare records with typed fields, and to store unboxed values in those fields, and to cause access to their values to emit boxing/unboxing instructions around them. The optimizer will get rid of those boxing/unboxing instructions if it can. Good luck!

textual CPS

The CPS language is key to all compiler work in Guile, but it doesn't have a nice textual form like LLVM IR does. Design one, and implement a parser and an unparser!

avoiding arity checks

If you know the procedure you are calling, like if it's lexically visible, then if you are calling it with the right number of arguments you can skip past the argument check and instead do a call-label directly into the body. Would be pretty neat!

unboxed calls and returns

Likewise if a function's callers are all known, it might be able to unbox its arguments or return value, if that's a good idea. Tricky! You could start with a type inference pass or so, and maybe that could produce some good debugging feedback too.

module-level inlining

Guile currently doesn't inline anything that's not lexically visible. Unfortunately this restriction extends to top-level definitions in a module: they are treated as mutable and so never inlined/optimized/etc. Probably we need to change the semantics here such that a module can be compiled as a unit, and all values which are never mutated can be assumed to be constant. Probably you also want a knob to turn off this behavior, but really you can always re-compile and re-load a module as a whole if re-loading a function at run-time doesn't work because it was inlined. Anyway. Some semantic work here, but some peval work as well. Be careful!

cross-module inlining

Likewise Guile currently doesn't inline definitions from other modules. However for small functions this really hurts. Guile should probably serialize tree-il for small definitions in .go files, and allow peval to speculatively inline imported definitions. This is related to the previous point and has some semantic implications.

bobobobobobonus! native compilation

Thinking realistically, native compilation is the next step. We have the object file format, cool. We will need the ability to call out from machine code in .go files to run-time functions, so we need to enhance the linker, possibly even with things like PLT/GOT sections to avoid dirtying too many pages. We need to lower the CPS even further, to get closer to some kind of machine model, then go specific, with an assembler for each architecture. The priority in the beginning will be simplicity and minimal complexity; good codegen will come later. This is obviously the most attractive thing but it's also the most tricky, design-wise. I want to do at least part of this, so though you can't have it all, you are welcome to help :)

That's it for now. I'll amend the post with more things as and when I think of them. Comments welcome too, as always. Happy hacking!

by Andy Wingo at February 04, 2016 09:38 PM

February 03, 2016

guile @ Savannah

GNU Guile 2.1.2 released (beta)

We are chuffed to announce GNU Guile release 2.1,2, the next pre-release in what will become the 2.2 stable series. This release features unboxed arithmetic and dramatically faster build times, along with a number of small speed and memory improvements.

See the original announcement at https://lists.gnu.org/archive/html/guile-user/2016-02/msg00022.html for full details.

by Andy Wingo at February 03, 2016 04:42 PM

FSF Blogs

Friday Free Software Directory IRC meetup: February 5th

Join the FSF and friends Friday, February 5th, from 12pm to 3pm EST (17:00 to 20:00 UTC) to help improve the Free Software Directory.

Participate in supporting the Free Software Directory by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on freenode.

Tens of thousands of people visit directory.fsf.org each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the Free Software Directory has been and continues to be a great resource to the world over the past decade, it has the potential of being a resource of even greater value. But it needs your help!

If you are eager to help and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly FSD Meetings pages that everyone is welcome to contribute to before, during, and after each meeting.

February 03, 2016 02:54 PM

GNUtls

GnuTLS 3.4.9

Released GnuTLS 3.3.21 and GnuTLS 3.4.9 which are bug fix releases in the previous and current stable branches. The former disables RC4 from the default priorities.

by Nikos Mavrogiannopoulos (nmav@gnutls.org) at February 03, 2016 12:00 AM

January 31, 2016

health @ Savannah

GNU Health 3.0.1 patchset released !

Dear community

GNU Health 3.0.1 patchset has been released !

Priority: High

Table of Contents

  • About GNU Health Patchsets
  • Summary of this patchset
  • Updating your system with the GNU Health control Center
  • Installation notes
  • List of issues related to this patchset

About GNU Health Patchsets

We provide "patchsets" to stable releases. Patchsets allow applying bug fixes and updates on production systems. Always try to keep your production system up-to-date with the latest patches.

Patches and Patchsets maximize uptime for production systems, and keep your system updated, without the need to do a whole installation.

For more information about GNU Health patches and patchsets you can visit https://en.wikibooks.org/wiki/GNU_Health/Patches_and_Patchsets

NOTE: Patchsets are applied on previously installed systems only. For new, fresh installations, download and install the whole tarball (ie, gnuhealth-3.0.1.tar.gz)

Summary of this patchset

  • The main issues resolved have to do with the age representation . There were issues with the generic timedelta method, so we reverted to the original relativedelta function, with deals better with ages. Timedelta (and its related Tryton fields.TimeDelta) is still used to show shorter periods of time, such as patient waiting times, duration of evaluations / surgeries, etc...
  • Bug fixes on reports, especially on dates, ages and date formats.
  • The gnuhealth-setup and and gnuhealth-control have been updated. These tools are also packaged separately, with their own numbering, so you can always download the latest version without the need of applying a patchset.

Refer to the List of issues related to this patchset for a comprehensive list of fixed bugs.

Updating your system with the GNU Health control Center

Starting GNU Health 3.x series, you can do automatic updates on the GNU Health and Tryton kernel and modules using the GNU Health control center program.

Please refer to the administration manual section ( https://en.wikibooks.org/wiki/GNU_Health/Control_Center )

The GNU Health control center works on standard installations (those done following the installation manual on wikibooks).

Installation Notes

You must apply previous patchsets before installing this patchset. If your patchset level is 3.0.0, then just follow the general instructions. You can find the patchsets at GNU Health FTP site (ftp://ftp.gnu.org/gnu/health/)

Follow the general instructions at

List of issues related to this patchset

#47031 gnuhealth-control status error when multiple instances are running
#47023 Patient CAGE score not available to admin group
#46996 vobject version name changed
#46991 health : gnuhealth.patient has no attribute "patient_age" at immunization status report
#46961 Use relative delta for age
#46953 Error on creating appointment if healthprof.name.internal_user == None
#46950 Error when generating the summary report
#46947 Traceback when generating the invoice associated to a prescription service
#46942 Prescription report : Age on prescription is shown in days
#46908 Use biological sex in view attributes

For detailed information about each issue, you can visit https://savannah.gnu.org/bugs/?group=health

by Luis Falcon at January 31, 2016 04:44 PM

January 30, 2016

FSF Blogs

Support the FSF for thirty more years

This post was written by FSF board member and co-founder of CivicActions Henry Poole.

But, as we imagine into the future, it is clear that operating systems are becoming embedded in everything. Our homes, our automobiles, and even our bodies. While a stand for free software was impersonal thirty years ago, in another thirty years it will be very personal. Operating systems will likely be embedded in our bodies, and the rights to use, study, modify, and share that software to stay alive and connect with others should be a fundamental human right. The last thirty years wasn't easy -- maintaining a state of integrity is often difficult and lonely. The next thirty years are likely to be just as difficult (or more).

Don't retire so young -- make a personal commitment today to become a lifetime FSF supporter. Stand with us for another thirty years. In the years ahead, people around the world will celebrate that stand.

And if you are fortunate enough to have a job at a company that uses free software, ask your company to increase patronage of free software by supporting the FSF. The FSF has a patron program for businesses that support and use free software. Of the 18,000 new US businesses that start every day, it would be great to see a new patron each day support the FSF.

January 30, 2016 03:30 PM

January 29, 2016

FSF Blogs

January 2016 - Stockholm University

On January 25th, 2016, RMS was back at Stockholm University's Aula Magna, where he had already spoken back in 2011,1 to give his speech “A Free Digital Society,” to an audience of over 800 people.

(Photos and poster under CC BY-SA 3.0. Photos courtesy of Kjell Ericson.)

Please fill out our contact form, so that we can inform you about future events in and around Stockholm and Östersund, both of which RMS visited on this trip to Sweden. Please see www.fsf.org/events for a full list of all of RMS's confirmed engagements, and contact rms-assist@gnu.org if you'd like him to come speak.

Thank you to Daniel Stenberg for having made this visit possible!


1. Kjell took some great photos of that speech, as well.

January 29, 2016 09:10 PM

Lonely Cactus

A Portrait of the Coder as an Old Man

I've been thinking quite a bit about being a older coder.  I've come to the realization that, like most other professions or activities, as we age, we need to make accommodations.

For example, as we age, the short term memory begins to fade.  Where once I could reliably recall the names and type details of hundreds of the variables, functions, and methods I've touched on a large task, now I can only reliably remember a couple dozen.  As a young man, I could focus on whatever function I was modifying, using memory and recall should I need to add a function call or variable reference.  Now I do spend a lot of time tap dancing through code, always referencing headers and declarations, with dozens of files open at any given moment.

On the flip side, the long term memory, now so richly loaded with associations, can get bogged down in those associations.  Every module or algorithm brings to mind another module or algorithm.  This is like that is like this is like that.  Which is very useful given that there is nothing new under the sun, but sometimes there is something new that needs to be considered on its own merits.

Also, there is the problem of irascibility.

I have spend almost all of my professional life coding in subsets of C.  Various embedded or safe or DO-178B restrictions to C.

C, in particular, has two problems for the old coder.

First is the problem of irascibility.

C is very simple with a minimal library.  I remember the first time I coded a doubly-linked list or a binary tree.  So satisfying to get this concept and see it function.  It is fun the first time, or the second.  But after 30 years, I never again want to code a doubly-linked list, a sort, a hash map, a binary tree, a dispatcher, a parser, a lexer, a matrix pseudoclass, a polynomial solver, or a Kalman filter.  It is just no fun anymore.

Second is the problem of verbosity.

Since I know I no longer have an infallible, photographic recall of an entire codebase, it helps to be able to have as much code on the screen at any given moment: to be able to see as much as possible of what I am doing.  And C, in particular, is so verbose that it restricts the amount of meaning I can throw up on any given screen.  C so often oscillates between being information dense and having a paucity of information per line.

So, then, coding as an old man...
  • I need my language and tools to help me remember details of variables, functions, and methods.  And I need those tools to help me remember in a way that doesn't destroy my flow.
  • I need my language and tools to let me see as much of what I am doing as possible.
  • I need my language and tools to not require me to re-code or re-integrate fundamental algorithms that are no longer entertaining.
  • I need my language and tools to help me determine if a code block is or is not a fundamental algorithm.  If it is, I want to use that fundamental algorithm.
Let's imagine, for a second, the properties of a theoretical programming language created for older coders. Such a theoretical language would be designed from the ground to play well with tools.  There should be nothing in the grammar of the language that would prevent tools from autocompleting or suggesting a given declaration.  The scoping and context rules should not have the sort of ambiguity that prevents effective tooling.  And compilation errors should be flagged immediately.

Also such a language should have a consistent level of information density, so that a screenful of text is sufficient to display a "unit" of code.

I don't often stray too far from the C-language family.  To be honest, C# is pretty good as an old man language. But, I've been trying to brush up on my C++17.

#include 〈algorithm〉 is pretty cool.

by Mike (noreply@blogger.com) at January 29, 2016 06:50 PM

FSF Blogs

We're so close! Will you help the FSF shine in 2016?

We've received nearly 6,000 donations and have signed up more than 400 new members during this fundraiser. Your donation or membership will help us make 2016 a shining year for software freedom.

FSF Patron organization CivicActions has pledged to match up to $2,500 for all general donations from patrons or individuals that come in before our winter fundraiser ends. Join as a Patron now and double your impact.

To those of you who have given, for the first time or many times, thank you. The Free Software Foundation relies heavily on individual donors in order to continue fighting for computer user rights. We couldn't do this without you.

As the fundraiser comes to a close, we are turning our attention to our goals for the coming year. We want to:

  • analyze and respond to the results of our Vision Survey (open until Monday, February 1 at 05:00 UTC)
  • make this year's LibrePlanet better than ever, kicking off with a remote conversation with NSA whistleblower Edward Snowden
  • launch a new, updated site to put the best face possible on free software
  • invest in more staff so we can expand the Respects Your Freedom certification program, bring the free software philosophy to more people, aid more free software development in high priority areas, and deftly enforce the GNU General Public License
  • keep up the momentum we've built in the fight against the Trans Pacific Partnership, the Digital Millennium Copyright Act's anti-circumvention rules, and the FCC-instigated lockdown of wireless devices
  • continue to campaign for the abolition of software patents
  • resist plans for an official Digital Restrictions Management extension to HTML5
  • create and improve upon free software advocacy materials, including videos
  • grow and update the FSF and GNU Project infrastructure, allowing us to accommodate more GNU projects
  • improve the FSF's development infrastructure, email, code hosting, and other services for GNU developers and FSF members, and incorporate more community members in FSF sysadmin and Web development projects

We need financial resources to get things done for you, but we also need your advice and guidance. Tell us how you feel about the free software movement and the work we do by responding to our survey by Sunday. Your compliments and criticisms will help us plan another thirty years of work.

We are so close to raising $450,000 by Sunday, January 31st, ensuring that we can make great things happen in 2016. Will you help us meet -- or even exceed -- that goal? Make a one-time donation now, or make a long-term commitment to supporting the FSF by becoming an associate member for as little as $10/month (student memberships are further discounted). Membership offers many great benefits, including gratis admission to LibrePlanet.

Read more about the recent accomplishments and upcoming goals of our campaigns, licensing, and tech teams.

January 29, 2016 06:05 PM

January 28, 2016

FSF Events

Richard Stallman - "El software libre en la ética y en la práctica" (Barcelona, Spain)

Richard Stallman hablará sobre las metas y la filosofía del movimiento del Software Libre, y el estado y la historia del sistema operativo GNU, el cual junto con el núcleo Linux, es actualmente utilizado por decenas de millones de personas en todo el mundo.

Esa charla de Richard Stallman no será técnica y será abierta al público; todos están invitados a asistir.

Lugar: Facultad de Bibliotecomania i documentació, called Melcior de Palau, 140, 08014 Barcelona, Spain

Favor de rellenar este formulario, para que podamos contactarle acerca de eventos futuros en la región de Barcelona.

January 28, 2016 11:35 AM

January 27, 2016

FSF News

Edward Snowden will kick off LibrePlanet 2016 in Cambridge, Massachusetts

The annual free software conference will kick off at Massachusetts Institute of Technology's (MIT) Stata Center in Cambridge, Massachusetts on the morning of Saturday, March 19th with "The last lighthouse: Free software in dark times", in which Snowden (who will appear via a free software live video stream) and Daniel Kahn Gillmor will discuss free software, surveillance, power, and control of the future.

Daiel Kahn Gillmor

Creative Commons License
Daniel Kahn Gillmor
This work by Daniel Kahn Gillmor is licensed under a Creative Commons Attribution ShareAlike 4.0 International License.

Edward Snowden

Creative Commons License
Edward Snowden
Screenshot of Citizen Four by Praxis Films.
by Laura Poitras is licensed under a Creative Commons Attribution 3.0

"We're thrilled and honored to be hosting this conversation. Edward Snowden has ignited desperately needed discussion around the world about the meaning of privacy, the power of governments and large corporations, and the impact of secretive technology on our freedom. I can't think of a more powerful way to launch this year's conference, and I can't wait to see what great things the LibrePlanet community of activists and developers will do with the energy," said FSF's executive director John Sullivan.

Edward Snowden is a former intelligence officer who served the Central Intelligence Agency (CIA), NSA, and Defense Intelligence Agency (DIA) for nearly a decade as a subject matter expert on technology and cybersecurity. In 2013, he revealed the NSA was unconstitutionally seizing the private records of billions of individuals who had not been suspected of any wrongdoing, resulting in the largest debate about reforms to US surveillance policy since 1978. Today, he works on methods of enforcing human rights through the application and development of new technologies. He joined the board of Freedom of the Press Foundation in February 2014.

Daniel Kahn Gillmor is a technologist with the ACLU's Speech, Privacy and Technology Project, and a free software developer. He's a Free Software Foundation Associate member, a member of Debian, a contributor to a wide range of free software projects, and a participant in protocol development standards organizations like the Internet Engineering Task Force (IETF), with an eye toward preserving and improving civil liberties and civil rights through our shared infrastructure.

For the third year in a row, LibrePlanet will be held at MIT's Stata Center in Cambridge, Massachusetts, on March 19th and 20th, 2016. Co-presented by the Free Software Foundation and MIT's Student Information Processing Board (SIPB), the rest of the LibrePlanet program will be announced soon.

Registration for LibrePlanet is now open. Admission to the conference is gratis for FSF members and students.

About the Free Software Foundation

The Free Software Foundation, founded in 1985, is dedicated to promoting computer users' right to use, study, copy, modify, and redistribute computer programs. The FSF promotes the development and use of free (as in freedom) software — particularly the GNU operating system and its GNU/Linux variants — and free documentation for free software. The FSF also helps to spread awareness of the ethical and political issues of freedom in the use of software, and its Web sites, located at fsf.org and gnu.org, are an important source of information about GNU/Linux. Donations to support the FSF's work can be made at https://donate.fsf.org. Its headquarters are in Boston, MA, USA.

More information about the FSF, as well as important information for journalists and publishers, is at https://www.fsf.org/press.

Media Contacts

Georgia Young
Program Manager
Free Software Foundation
+1 (617) 542 5942
georgia@fsf.org

January 27, 2016 08:50 PM

FSF Blogs

Edward Snowden will kick off LibrePlanet 2016: Will you be there?

Be sure to join us the morning of Saturday, March 19th for "The last lighthouse: Free software in dark times" in which Snowden (who will appear via a free software video call) and Daniel Kahn Gillmor will discuss free software, surveillance, power, and control of the future.

Daniel Kahn Gillmor

Creative Commons License
Daniel Kahn Gillmor
This work by Daniel Kahn Gillmor is licensed under a Creative Commons Attribution ShareAlike 4.0 International License.

Edward Snowden

Creative Commons License
Edward Snowden
Screenshot of Citizen Four by Praxis Films.
by Laura Poitras is licensed under a Creative Commons Attribution 3.0

Edward Snowden is a former intelligence officer who served the Central Intelligence Agency (CIA), NSA, and Defense Intelligence Agency (DIA) for nearly a decade as a subject matter expert on technology and cybersecurity. In 2013, he revealed the NSA was unconstitutionally seizing the private records of billions of individuals who had not been suspected of any wrongdoing, resulting in the largest debate about reforms to US surveillance policy since 1978. Today, he works on methods of enforcing human rights through the application and development of new technologies. He joined the board of Freedom of the Press Foundation in February 2014.

Daniel Kahn Gillmor is a technologist with the ACLU's Speech, Privacy and Technology Project, and a free software developer. He's a Free Software Foundation Associate member, a member of Debian, a contributor to a wide range of free software projects, and a participant in protocol development standards organizations like the Internet Engineering Task Force (IETF), with an eye toward preserving and improving civil liberties and civil rights through our shared infrastructure.

LibrePlanet will once again be held at Massachusetts Institute of Technology's (MIT) Stata Center in Cambridge, Massachusetts, March 19th and 20th, 2016. The continued partnership of the Free Software Foundation and MIT's Student Information Processing Board (SIPB) means another unmissable gathering of free software enthusiasts. The rest of the LibrePlanet program, including other fantastic keynotes, workshops, and sessions, will be announced soon.

You don't want to miss this. Register for LibrePlanet today to ensure admission -- conference capacity is limited and we may have to close registration. As always, admission to the conference is gratis for FSF members and students. Not a member? Join today for gratis admission to LibrePlanet -- plus, you'll help us meet our winter fundraising goal of $450,000 by January 31st.

Does your organization or company want to exhibit at LibrePlanet 2016? Apply to be an exhibitor by January 31st to reach hundreds of hackers, activists, and free software enthusiasts at the Stata Center.

January 27, 2016 08:50 PM

Friday Free Software Directory IRC meetup: January 29th

Participate in supporting the Free Software Directory by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on freenode.

Tens of thousands of people visit directory.fsf.org each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the Free Software Directory has been and continues to be a great resource to the world over the past decade, it has the potential of being a resource of even greater value. But it needs your help!

If you are eager to help and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly FSD Meetings pages that everyone is welcome to contribute to before, during, and after each meeting.

January 27, 2016 02:30 PM

January 26, 2016

Andreas Enge

Novena board set-up for the GNU Guix build farm, part 2

novena
Now that the new Novena board donated by Bunnie of Kosagi (Sutajio Ko-Usagi Pte Ltd) to the GNU Guix project is set up physically, it is time to install the necessary software for using it as a Guix build machine and to integrate it into the hydra build farm.

Add an external hard drive

The board as delivered comes with a 4GB micro-sd card, on which the Debian operating system is preinstalled. This could easily be replaced by a larger micro-sd card. But for a Guix build machine, the more space in /gnu to store the compiled packages, the better, so we are adding an external hard disk. Also, /tmp needs to contain a few gigabytes, since that is where packages are built; in normal operation, this is very sensibly set up as a tmpfs, but the 4GB of main memory are simply not enough for us. I am not sure whether a micro-sd card or an external conventional hard disk would be faster; an ssd would be best, but I do not have a spare one. So we will make do with an external usb hard disk of 1TB that I have lying around. Normally I like to encrypt my file systems. Here this seems like unnecessary work: The system is not going to travel and will contain only public data, namely compiled Guix packages. Once attached, the disk shows an ext4-formatted partition under /dev/sda1. Just recently I learnt about bind-mounting external directories into local ones, so we are going (as root) to create two directories gnu (to contain the store) and tmp (to have some space in which the packages can be built) in the partition and mount them as the corresponding directories on the Novena board.

mount /dev/sda1 /media
mkdir /media/gnu
mkdir /media/tmp

The following three lines are added to /etc/fstab:

/dev/sda1  /media ext4 defaults 0 2
/media/gnu /gnu   none bind     0 0
/media/tmp /tmp   none bind     0 0

and the line defining /tmp as a tmpfs needs to be deleted or commented out, since otherwise it seems to have precedence. (Well, this is almost the truth; instead of /dev/sda1, I used /dev/disk/by-id/wwn-0x50004cf209015f7c-part1, where the last part depends on the disk.) Remounting something over /tmp makes me feel uneasy, so it is maybe safer to reboot. And here we go:

df -h /tmp /gnu
/dev/sda1       917G   72M  871G   1% /tmp
/dev/sda1       917G   72M  871G   1% /gnu

It has worked!

Install Guix and the Guix daemon

The easiest approach to installing Guix is the binary installation method; we simply follow the steps given in the manual.

  1. Download the tarball

    cd /tmp
    wget ftp://alpha.gnu.org/gnu/guix/guix-binary-0.9.0.armhf-linux.tar.xz,
    

    its signature file

    wget ftp://alpha.gnu.org/gnu/guix/guix-binary-0.9.0.x86_64-linux.tar.xz.sig
    

    and check the signature

    gpg guix-binary-0.9.0.armhf-linux.tar.xz.sig
    
  2. As root, unpack the tarball and install the files.

    cd /tmp
    tar --warning=no-timestamp -xf guix-binary-0.9.0.armhf-linux.tar.xz
    mv var/guix /var
    mv gnu/store /gnu
    
  3. The tarball contains a prefabricated profile for root containing the guix package itself; make it available in the usual location:

    ln -sf /var/guix/profiles/per-user/root/guix-profile \
           ~root/.guix-profile
    

    So that root finds the Guix commands, add the following line to .bashrc:

    source /root/.guix-profile/etc/profile
    

    The file in question contains a number of search paths definitions; in particular, it augments the PATH variable to point to the bin and sbin subdirectories of the profile. A nice feature of this approach is that the sourced file is part of the profile itself; so it gets updated together with the profile.

  4. Create groups and users for the Guix daemon and the package builders:

    groupadd --system guixbuild
    for i in `seq -w 1 10`; do
      useradd -g guixbuild -G guixbuild           \
              -d /var/empty -s `which nologin`    \
              -c "Guix build user $i" --system    \
              guixbuilder$i;
    done
    
  5. Run the daemon. There is a service file for systemd, which for some unknown reason does not work for me; instead, I simply run the daemon in a screen session:

    screen
    guix-daemon --build-users-group=guixbuild
    <Ctrl>-<a>-<d>
    
  6. Make the guix command available for others.

    cd /usr/local/bin
    ln -s /var/guix/profiles/per-user/root/guix-profile/bin/guix
    
  7. Authorise substitutes from hydra:

    guix archive --authorize \
      < ~root/.guix-profile/share/guix/hydra.gnu.org.pub
    
  8. Test that everything works:

    guix package -i screen
    

    This command downloads the screen package (together with some more basic packages) from hydra.gnu.org and installs it into the root profile. The next time screen is run, it will be the one from GNU Guix! (But beware, it will not connect to the running screen session started with the Debian binary. So it might be safer to stop the latter and to restart the daemon in a Guix screen session.)

Set up the build machine and integrate it into the build farm

This step is called offloading, and it is also described in more or less detail in the manual.

  1. On the machine doing the offloading of package build jobs (called hydra in the following), edit the file /etc/guix/machines.scm and add a definition for the new build machine (called novena in the following) and return it as a list of build machines (this last line is commented out, as we first need to complete the following steps):

    (define novena
      (build-machine
        (name "novena.enge.fr")
        (port 22)
        (user "hydra")
        (system "armhf-linux")
        (private-key "/home/hydra/.lsh/identity")
        (ssh-options '("-c" "aes256-ctr"))
        (speed 1.0)
        (parallel-builds 2)))
    
    ;; (list novena)
    

    The name field contains the host name of novena, the port field the port under which the ssh server accepts connections (22 is the default and can be omitted), the user field gives the user name under which hydra connects, the private-key field gives the file on hydra containing the private key used to connect. Some subtleties arise when the build machine (as in our case) runs the openssh daemon, since hydra connects using GNU lsh. Both speak the standard ssh protocol and can thus talk to each other, but have different defaults and key file formats. The ssh-options are needed to make lsh use an encryption algorithm that is also used by recent openssh versions. We will see more back and fro between lsh and openssh in the following.

  2. Create and mutually authorise key pairs in Guix, to allow the two machines to exchange store items via guix archive: hydra will send build inputs to novena, and novena will send compiled output packages back.

    On hydra, run (as root)

    guix archive --generate-key
    

    to generate a key pair (this can take a very long time, supposedly because there is not much entropy to gather on non-desktop machines). Copy the public key in /etc/guix/signing-key.pub over to an arbitrary place on novena, and authorise it (also as root) via the command

    guix archive --authorize < signing-key.pub
    

    already seen above. (The authorised key should now be listed in /etc/guix/acl.) Symmetrically, create a key pair on novena and authorise it on hydra.

  3. On novena, create (as root) the password-less user hydra under which hydra connects:

    adduser --disabled-password hydra
    

    Mutually authorise public ssh keys as follows, so that hydra can connect as the user hydra to novena.

    To authorise the hydra key on novena, first convert the public key corresponding to the private key in /home/hydra/.lsh/identity (specified above in /etc/guix/machines.scm on hydra) to the openssh format. So on hydra, execute the following command:

    lsh-export-key --openssh < /home/hydra/.lsh/identity.pub
    

    The output (starting with ssh-rsa) needs to be copied by the user hydra on novena into /home/hydra/.ssh/authorized_keys.

    Conversely, teach hydra to accept the server key of novena, using the TOFU – Trust On First Use paradigm. As root on hydra, execute

    lsh -c aes256-ctr -i /home/hydra/.lsh/identity -l hydra -p 22 \
      -q --sloppy-host-authentication \
      novena.enge.fr
    

    where the different command line items correspond to the fields defined in /etc/guix/machines.scm as seen above, and -q --sloppy-host-authentication instructs lsh to silently accept the public server key of novena. This command should have connected you to novena; disconnect again. The file /root/.lsh/captured_keys should now contain as its last line the server key of novena in lsh format, enclosed in brackets {}. Copy this line into /root/.lsh/host-acls. Now the connection should be possible without the -q --sloppy-host-authentication stance.

  4. On novena, the guix command has already been made available above. Moreover, the Guix modules need to be made available to guile. For this, first install guile on novena (as user hydra):

    guix package -i guile
    

    Then add the following three lines to the top of .bashrc (why at the top and not the bottom is Debian specific and would merit another blog post):

    source $HOME/.guix-profile/etc/profile
    export GUILE_LOAD_PATH=\
    /var/guix/profiles/per-user/root/guix-profile/share/guile/site/2.0/
    export GUILE_LOAD_COMPILED_PATH=\
    /var/guix/profiles/per-user/root/guix-profile/share/guile/site/2.0/
    

    To test whether this was successful, execute as root on hydra the following command:

    lsh -c aes256-ctr -i /home/hydra/.lsh/identity -l hydra -p 22 \
      novena.enge.fr \
      guile -c "'(use-modules (guix config))'"
    

    If everything is okay, then the command outputs nothing; otherwise, it will probably output an error message ending in

    ERROR: In procedure scm-error:
    ERROR: no code for module (guix config)
    
  5. We need to create and make world-writable an additional directory on novena:

    mkdir /var/guix/gcroots/tmp
    chmod a+w /var/guix/gcroots/tmp
    
  6. Finally, uncomment the last line of /etc/guix/machines.scm on hydra so that it reads

    (list novena)
    

Now the build machine is ready for work!

by andreas at January 26, 2016 08:10 PM

FSF Blogs

Key charities that advance software freedom are worthy of your urgent support

I've had the pleasure and the privilege, for the last 20 years, to be either a volunteer or employee of the two most important organizations for the advancement of software freedom and users' rights to copy, share, modify and redistribute software. In 1996, I began volunteering for the FSF and worked as its Executive Director from 2001–2005. I continued as a volunteer for the FSF since then, and now serve as a volunteer on FSF's Board of Directors. I was also one of the first volunteers for Software Freedom Conservancy when we founded it in 2006, and I was the primary person doing the work of the organization as a volunteer from 2006–2010. I've enjoyed having a day job as a Conservancy employee since 2011.

These two organizations have been the center of my life's work. Between them, I typically spend 50–80 hours every single week doing a mix of paid and volunteer work. Both my hobby and my career are advancing software freedom.

I choose to give my time and work to these organizations because they provide the infrastructure that make my work possible. The free software community has shown that the work of many individuals, who care deeply about a cause but cooperate together toward a common goal, has an impact greater than any individuals can ever have working separately. The same is often true for cooperating organizations: charities, like Conservancy and the FSF, that work together with each other amplify their impact beyond the expected.

Both Conservancy and the FSF pursue specific and differing approaches and methods to the advancement of software freedom. The FSF is an advocacy organization that raises awareness about key issues that impact the future of users' freedoms and rights, and finds volunteers and pays staff to advocate about these issues. Conservancy is a fiscal sponsor, which means one of our key activities is operational work, meeting the logistical and organizational needs of volunteers so they can focus on the production of great free software and free documentation. Meanwhile, both Conservancy and FSF dedicate themselves to sponsoring software projects: the FSF through the GNU Project, and Conservancy through its member projects. And, most importantly, both charities stand up for the rights of users by enforcing and defending copyleft licenses such as the GNU General Public License (GPL).

Conservancy and the FSF show in concrete terms that two charities can work together to increase their impact. Last year, our organizations collaborated on many projects, such as the proposed FCC rule changes for wireless devices, jointly handled a GPL enforcement action against Canonical, Ltd., published the principles of community-oriented GPL enforcement, and continued our collaboration on copyleft.org. We're already discussing lots of ways that the two organizations can work together in 2016!

I'm proud to give so much of my time and energy to both these excellent organizations. But, I also give my money as well: I was the first person in history to become an Associate Member of the FSF (back in November 2002), and have gladly paid my monthly dues since then. Today, I also signed up as an annual Supporter of Conservancy, because I want to ensure that Conservancy meets its current pledge match — the next 215 Supporters who sign up before January 31st will double their donation via the match.

For just US$20 each month, you make sure the excellent work of both these organizations can continue. This is quite a deal: if you are an employed, University-educated professional living in the industrialized world, US$20 is probably the same amount you'd easily spend on a meals at restaurants or other luxuries. Isn't it even a better luxury to know that these two organizations can have a years' worth of effort of standing up for your software freedom in 2016? You can make the real difference by making your charitable contribution to these two organizations today:

Please don't wait: both fundraising deadlines are just five days away!

January 26, 2016 04:33 PM

Vision Survey closes this week. Add your voice!

The survey takes only five to fifteen minutes to complete. Fill it out now!

It's important that this survey reach a large and diverse range of people who care about free software. Please share it by whatever means will reach your friends best — social media, email, IRC, or word of mouth.

One more thing: Last week we released a video of community members telling their free software stories. Watch it on fsf.org and if you like telling stories, consider sharing your own on social media (hashtag #freesoftwarestory).

January 26, 2016 03:58 PM