Planet GNU

Aggregation of development blogs from the GNU Project

July 21, 2024

parallel @ Savannah

GNU Parallel 20240722 ('Assange') released [stable]

GNU Parallel 20240722 ('Assange') has been released. It is available for download at: lbry://@GnuParallel:4

Quote of the month:

  parallel is frickin great for launching jobs on multiple
  machines. Ansible and Jenkins and others may be good too but I was
  able to jump right in with parallel.
    -- dwhite21787@reddit
 
New in this release:

  • No new features. This is a candidate for a stable release.
  • Bug fixes and man page updates.


News about GNU Parallel:


GNU Parallel - For people who live life in the parallel lane.

If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.


About GNU Parallel


GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
    12345678 883c667e 01eed62f 975ad28b 6d50e22a
    $ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
    cc21b4c9 43fd03e9 3ae1ae49 e28573c0
    $ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
    79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
    fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
  • Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference


If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)


If GNU Parallel saves you money:



About GNU SQL


GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.


About GNU Niceload


GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

21 July, 2024 03:01AM by Ole Tange

July 19, 2024

GNU Taler news

Video interview with Mikolai Gütschow on payments for the Internet of Things

On the occasion of the Point Zero Forum's Innovation Tour, Evgeny Grin has interviewed Mikolai Gütschow who designed and implemented solutions for the payments in the Internet of Things (IoT).

19 July, 2024 08:52AM

July 18, 2024

Video interview with Isidor Walliman, creator of the Netzbon regional currency in Basel

On the occasion of the Point Zero Forum's Innovation Tour, Evgeny Grin has interviewed Isidor Wallimann who is introducing GNU Taler for the local currency Netzbon in Basel.

18 July, 2024 10:00PM

GNUnet News

The European Union must keep funding free software

The European Union must keep funding free software

The GNUnet project was granted NGI funding via NLnet . Other FOSS related projects also benefit from NGI funding. This funding is now at risk for future projects.

The following is an open letter initially published in French by the Petites Singularités association. To co-sign it, please publish it on your website in your preferred language, then add yourself to this table .

Open Letter to the European Commission.

Since 2020, Next Generation Internet ( NGI ) programmes, part of European Commission’s Horizon programme, fund free software in Europe using a cascade funding mechanism (see for example NLnet’s calls ). This year, according to the Horizon Europe working draft detailing funding programmes for 2025, we notice that Next Generation Internet is not mentioned any more as part of Cluster 4.

NGI programmes have shown their strength and importance to supporting the European software infrastructure, as a generic funding instrument to fund digital commons and ensure their long-term sustainability. We find this transformation incomprehensible, moreover when NGI has proven efficient and economical to support free software as a whole, from the smallest to the most established initiatives. This ecosystem diversity backs the strength of European technological innovation, and maintaining the NGI initiative to provide structural support to software projects at the heart of worldwide innovation is key to enforce the sovereignty of a European infrastructure. Contrary to common perception, technical innovations often originate from European rather than North American programming communities, and are mostly initiated by small-scaled organizations.

Previous Cluster 4 allocated 27 million euros to:

  • “Human centric Internet aligned with values and principles commonly shared in Europe” ;
  • “A flourishing internet, based on common building blocks created within NGI, that enables better control of our digital life” ;
  • “A structured ecosystem of talented contributors driving the creation of new internet commons and the evolution of existing internet commons”.

In the name of these challenges, more than 500 projects received NGI funding in the first 5 years, backed by 18 organisations managing these European funding consortia.

NGI contributes to a vast ecosystem, as most of its budget is allocated to fund third parties by the means of open calls, to structure commons that cover the whole Internet scope - from hardware to application, operating systems, digital identities or data traffic supervision. This third-party funding is not renewed in the current program, leaving many projects short on resources for research and innovation in Europe.

Moreover, NGI allows exchanges and collaborations across all the Euro zone countries as well as “widening countries” 1 , currently both a success and an ongoing progress, likewise the Erasmus programme before us. NGI also contributes to opening and supporting longer relationships than strict project funding does. It encourages implementing projects funded as pilots, backing collaboration, identification and reuse of common elements across projects, interoperability in identification systems and beyond, and setting up development models that mix diverse scales and types of European funding schemes.

While the USA, China or Russia deploy huge public and private resources to develop software and infrastructure that massively capture private consumer data, the EU can’t afford this renunciation. Free and open source software, as supported by NGI since 2020, is by design the opposite of potential vectors for foreign interference. It lets us keep our data local and favors a community-wide economy and know-how, while allowing an international collaboration. This is all the more essential in the current geopolitical context: the challenge of technological sovereignty is central, and free software allows addressing it while acting for peace and sovereignty in the digital world as a whole.


  1. As defined by Horizon Europe, widening Member States are Bulgaria, Croatia, Cyprus, Czechia, Estonia, Greece, Hungary, Latvia, Lituania, Malta, Poland, Portugal, Romania, Slovakia, and Slovenia. Widening associated countries (under condition of an association agreement) include Albania, Armenia, Bosnia, Feroe Islands, Georgia, Kosovo, Moldavia, Montenegro, Morocco, North Macedonia, Serbia, Tunisia, Turkeye, and Ukraine. Widening overseas regions are Guadeloupe, French Guyana, Martinique, Reunion Island, Mayotte, Saint-Martin, The Azores, Madeira, the Canary Islands. ↩︎

18 July, 2024 10:00PM

July 17, 2024

GNU Taler news

Video interview with Christian Blättler on his work on tokens for unlinkable discounts and subscriptions

On the occasion of the Point Zero Forum's Innovation Tour, Berna Alp has interviewed Christian Blättler who implemented a system for using GNU Taler for unlikable discounts and subscriptions.

17 July, 2024 10:00PM

health @ Savannah

MyGNUHealth 2.2.1 released

Dear community

I am happy to announce patchset 2.2.1 for MYGNUHealth, the GNU Health Personal Health Record.

This patchset fixes the following issues:


You can download MyGNUHealth source code from the official GNU Savannah (https://ftp.gnu.org/gnu/health/mygnuhealth/). You can also install MyGH from the Python Package Index (PyPI) or from your operating system distribution.

Happy hacking
Luis

17 July, 2024 10:10AM by Luis Falcon

July 16, 2024

GNU Taler news

Video interview with Nic Eigel, co-author of the GNU Taler real-time auditor

On the occasion of the Point Zero Forum's Innovation Tour, Berna Alp has interviewed Nicola Eigel who implemented a real-time auditor for the GNU Taler exchange with his colleague Cédric Zwahlen.

16 July, 2024 10:00PM

FSF Blogs

GNU Taler news

Video interview with Özgur Kesim on age-restricted digital cash

On the occasion of the Point Zero Forum's Innovation Tour, Evgeny Grin has interviewed Özgur Kesim who designed and implemented an age restricition mechanism inside the GNU Taler coins.

16 July, 2024 06:30PM

tasklist @ Savannah

Cleaning out old jobs

When I opened this Savannah project I imported items from the old GNU tasklist document. 20 years later all of the context has been lost (if there ever was any) so now if anyone asks about these tasks it just leads to frustration on everyone's part.

I therefore deleted the original help wanted entries that date back to 2003. If anyone wants to help the GNU project, the best way to do that is to pick one of the FSF's High-Priority projects:

https://www.fsf.org/campaigns/priority-projects

16 July, 2024 02:35PM by toby cabot

July 15, 2024

FSF Events

Free Software Directory meeting on IRC: Friday, July 19, starting at 12:00 EDT (16:00 UTC)

Join the FSF and friends on Friday, July 19 from 12:00 to 15:00 EDT (16:00 to 19:00 UTC) to help improve the Free Software Directory.

15 July, 2024 04:00AM

July 14, 2024

automake @ Savannah

July 13, 2024

gnuastro @ Savannah

Gnuastro 0.23 released

The 23rd release of GNU Astronomy Utilities (Gnuastro) is now available. See the full announcement for all the new features in this release and the many bugs that have been found and fixed: https://lists.gnu.org/archive/html/info-gnuastro/2024-07/msg00001.html

13 July, 2024 11:01PM by Mohammad Akhlaghi

July 10, 2024

FSF Blogs

July 09, 2024

Simon Josefsson

Towards Idempotent Rebuilds?

After rebuilding all added/modified packages in Trisquel, I have been circling around the elephant in the room: 99% of the binary packages in Trisquel comes from Ubuntu, which to a large extent are built from Debian source packages. Is it possible to rebuild the official binary packages identically? Does anyone make an effort to do so? Does anyone care about going through the differences between the official package and a rebuilt version? Reproducible-build.org‘s effort to track reproducibility bugs in Debian (and other systems) is amazing. However as far as I know, they do not confirm or deny that their rebuilds match the official packages. In fact, typically their rebuilds do not match the official packages, even when they say the package is reproducible, which had me surprised at first. To understand why that happens, compare the buildinfo file for the official coreutils 9.1-1 from Debian bookworm with the buildinfo file for reproducible-build.org’s build and you will see that the SHA256 checksum does not match, but still they declare it as a reproducible package. As far as I can tell of the situation, the purpose of their rebuilds are not to say anything about the official binary build, instead the purpose is to offer a QA service to maintainers by performing two builds of a package and declaring success if both builds match.

I have felt that something is lacking, and months have passed and I haven’t found any project that address the problem I am interested in. During my earlier work I created a project called debdistreproduce which performs rebuilds of the difference between two distributions in a GitLab pipeline, and display diffoscope output for further analysis. A couple of days ago I had the idea of rewriting it to perform rebuilds of a single distribution. A new project debdistrebuild was born and today I’m happy to bless it as version 1.0 and to announces the project! Debdistrebuild has rebuilt the top-50 popcon packages from Debian bullseye, bookworm and trixie, on amd64 and arm64, as well as Ubuntu jammy and noble on amd64, see the summary status page for links. This is intended as a proof of concept, to allow people experiment with the concept of doing GitLab-based package rebuilds and analysis. Compare how Guix has the guix challenge command.

Or I should say debdistrebuild has attempted to rebuild those distributions. The number of identically built packages are fairly low, so I didn’t want to waste resources building the rest of the archive until I understand if the differences are due to consequences of my build environment (plain apt-get build-dep followed by dpkg-buildpackage in a fresh container), or due to some real difference. Summarizing the results, debdistrebuild is able to rebuild 34% of Debian bullseye on amd64, 36% of bookworm on amd64, 32% of bookworm on arm64. The results for trixie and Ubuntu are disappointing, below 10%.

So what causes my rebuilds to be different from the official rebuilds? Some are trivial like the classical problem of varying build paths, resulting in a different NT_GNU_BUILD_ID causing a mismatch. Some are a bit strange, like a subtle difference in one of perl’s headers file. Some are due to embedded version numbers from a build dependency. Several of the build logs and diffoscope outputs doesn’t make sense, likely due to bugs in my build scripts, especially for Ubuntu which appears to strip translations and do other build variations that I don’t do. In general, the classes of reproducibility problems are the expected. Some are assembler differences for GnuPG’s gpgv-static, likely triggered by upload of a new version of gcc after the original package was built. There are at least two ways to resolve that problem: either use the same version of build dependencies that were used to produce the original build, or demand that all packages that are affected by a change in another package are rebuilt centrally until there are no more differences.

The current design of debdistrebuild uses the latest version of a build dependency that is available in the distribution. We call this a “idempotent rebuild“. This is usually not how the binary packages were built originally, they are often built against earlier versions of their build dependency. That is the situation for most binary distributions.

Instead of using the latest build dependency version, higher reproducability may be achieved by rebuilding using the same version of the build dependencies that were used during the original build. This requires parsing buildinfo files to find the right version of the build dependency to install. We believe doing so will lead to a higher number of reproducibly built packages. However it begs the question: can we rebuild that earlier version of the build dependency? This circles back to really old versions and bootstrappable builds eventually.

While rebuilding old versions would be interesting on its own, we believe that is less helpful for trusting the latest version and improving a binary distribution: it is challenging to publish a new version of some old package that would fix a reproducibility bug in another package when used as a build dependency, and then rebuild the later packages with the modified earlier version. Those earlier packages were already published, and are part of history. It may be that ultimately it will no longer be possible to rebuild some package, because proper source code is missing (for packages using build dependencies that were never part of a release); hardware to build a package could be missing; or that the source code is no longer publicly distributable.

I argue that getting to 100% idempotent rebuilds is an interesting goal on its own, and to reach it we need to start measure idempotent rebuild status.

One could conceivable imagine a way to rebuild modified versions of earlier packages, and then rebuild later packages using the modified earlier packages as build dependencies, for the purpose of achieving higher level of reproducible rebuilds of the last version, and to reach for bootstrappability. However, it may be still be that this is insufficient to achieve idempotent rebuilds of the last versions. Idempotent rebuilds are different from a reproducible build (where we try to reproduce the build using the same inputs), and also to bootstrappable builds (in which all binaries are ultimately built from source code). Consider a cycle where package X influence the content of package Y, which in turn influence the content of package X. These cycles may involve several packages, and it is conceivable that a cycle could be circular and infinite. It may be difficult to identify these chains, and even more difficult to break them up, but this effort help identify where to start looking for them. Rebuilding packages using the same build dependency versions as were used during the original build, or rebuilding packages using a bootsrappable build process, both seem orthogonal to the idempotent rebuild problem.

Our notion of rebuildability appears thus to be complementary to reproducible-builds.org’s definition and bootstrappable.org’s definition. Each to their own devices, and Happy Hacking!

Addendum about terminology: With “idempotent rebuild” I am talking about a rebuild of the entire operating system, applied to itself. Compare how you build the latest version of the GNU C Compiler: it first builds itself using whatever system compiler is available (often an earlier version of gcc) which we call step 1. Then step 2 is to build a copy of itself using the compiler built in step 1. The final step 3 is to build another copy of itself using the compiler from step 2. Debian, Ubuntu etc are at step 1 in this process right now. The output of step 2 and step 3 ought to be bit-by-bit identical, or something is wrong. The comparison between step 2 and 3 is what I refer to with an idempotent rebuild. Of course, most packages aren’t a compiler that can compile itself. However entire operating systems such as Trisquel, PureOS, Ubuntu or Debian are (hopefully) a self-contained system that ought to be able to rebuild itself to an identical copy. Or something is amiss. The reproducible build and bootstrappable build projects are about improve the quality of step 1. The property I am interested is the identical rebuild and comparison in step 2 and 3. I feel the word “idempotent” describes the property I’m interested in well, but I realize there may be better ways to describe this. Ideas welcome!

09 July, 2024 10:16PM by simon

July 05, 2024

FSF Blogs

July 03, 2024

Greg Casamento

What Apple has forgotten...

 When NeXT still existed and the black hardware was a thing, Steve Jobs made the announcement that OPENSTEP would be created and that the object model, not the operating system and not the hardware, was the important thing.

This is a concept that Apple has forgotten.  With it's push towards Apple Silicon and a walled-garden, Apple has committed itself to the same pitfall that NeXT fell into.  NeXT lacked the infrastructure to handle OPENSTEP running on multiple kinds of hardware, but the object model on different OSes was successful... this is evident in OPENSTEP1.1 for Solaris and OPENSTEP for NT.

GNUstep attempts to reach the same goal, but provides the APIs that are available with Cocoa.   The object model IS the important thing and this is why GNUstep is so important.  It breaks the walled garden and makes it possible for users to run their apps and tools on other operating systems.  GNUstep HASN'T forgotten and we believe this is a core concept that Apple has left behind.

03 July, 2024 11:03PM by Unknown (noreply@blogger.com)

July 02, 2024

FSF Blogs

direvent @ Savannah

GNU Direvent Version 5.4

GNU direvent version 5.4 is available for download.

New in this version:

Simultaneous execution limits


It is possible to limit number of command instances that are allowed to run simultaneously for a particular watcher.  This is done using
the max-instances statement in watcher section.

Restore the "nowait" default


In previous version, watchers waited for the handler to terminate, unless given the nowait option explicitly.  It is now fixed and nowait is the default, as described in the documentation.

Fix bug in generic to system event translation


Fix sentinel code


In some cases setting the sentinel effectively removed the original watcher.  That happened if the full file name of the original watcher
and its directory part produced the same hash code.

02 July, 2024 04:00PM by Sergey Poznyakoff

gdbm @ Savannah

GNU dbm version 1.24

GNU dbm version 1.24 is available for download. New in this version:

New gdbm_load option: --update


The --update (-U) option instructs gdbm_load to update an existing database.

Fix semantics of gdbm_load -r


The --replace (-r) is valid only when used together with --update.

Use getline in gdbmtool shell


New function: gdbm_load_from_file_ext


In contrast to gdbm_load and gdbm_load_from_file, which derive the value of the flag parameter for gdbm_open from the value of their replace argument, this function allows the caller to specify it explicitly. 

Bugfixes


  • Fix binary dump format for key and/or data of zero size (see bug 656)
  • Fix location tracking and recover command in gdbtool (see bug 566)
  • Fix possible buffer underflow in gdbmload.
  • Ensure any padding bytes in avail_elem structure are filled with 0. This fixes debian bug 1031276.
  • Improve the documentation.

02 July, 2024 02:28PM by Sergey Poznyakoff

July 01, 2024

Parabola GNU/Linux-libre

restart sshd immediately after upgrade

from arch:

After upgrading to openssh-9.8p1, the existing SSH daemon will be unable to accept new connections. When upgrading remote hosts, please make sure to restart the sshd service using systemctl try-restart sshd right after upgrading.

We are evaluating the possibility to automatically apply a restart of the sshd service on upgrade in a future release of the openssh-9.8p1 package.

01 July, 2024 06:52PM by bill auger

FSF Blogs

June 30, 2024

poke @ Savannah

GNU poke 4.2 released

I am happy to announce a new release of GNU poke, version 4.2.

This is a bugfix release in the 4.x series.

See the file NEWS in the distribution tarball for a list of issues
fixed in this release.

The tarball poke-4.2.tar.gz is now available at
https://ftp.gnu.org/gnu/poke/poke-4.2.tar.gz.

    > GNU poke (http://www.jemarch.net/poke) is an interactive, extensible
    > editor for binary data.  Not limited to editing basic entities such
    > as bits and bytes, it provides a full-fledged procedural,
    > interactive programming language designed to describe data
    > structures and to operate on them.


Thanks to the people who contributed with code and/or documentation to
this release.

Happy poking!

Mohammad-Reza Nabipoor

30 June, 2024 09:03PM by Mohammad-Reza Nabipoor

June 27, 2024

GNU Health

Migrar, migrant, migràrem

The title of this article, “Migrar, migrant, migràrem“, comes from a beautiful poem written by Laia Porcar[1], that inspired the strikingly profound painting by Sara Belles [2] “Jo per tu, fill meu“. The artists reflect the migrants ordeal to provide a better life to their children and families, even at the cost of losing their own lives.

GNU Health[3] is a Social project with some technology behind and the mission at Sea-Eye is one of the best examples. After all, GNU Solidario[4] is a NGO that focuses in the advancement of Social Medicine.

We live a world of injustice. Concentration of power, social gradient and poverty rates keep on the rise. Artificial intelligence is on the hands of mega private corporations, targeting our privacy and feeding the macabre business of war. The fight for scarce natural resources such as lithium or coltan creates coups in impoverished countries. Nature and non-human animals are used and abused as mere commodities. Our world turns a blind eye to the systematic crushing and eradication of civilian population by powerful armies. As a result, we live in a world where migration is not a choice, but the only way out for millions of human beings, even at the risk of becoming anonymous victims in the Atlantic ocean or Mediterranean sea mass graveyards.

“Jo per tu, fill meu”, by Sara Belles

But there is hope. The Sea-Eye mission is the end result of a network of solidarity, cooperation and empathy. The Free Software movement started by Richard Stallman[5]; Julian Sassenscheidt message in Mastodon and his presentation at GNU Health Con 2023[6] ; The work of our representative in Germany, Gerald Wiese; the Chaos Computer Club[7]; the team from L’Aurora[8] providing logistic support to the Search and Rescue vessels; the phenomenal Sea-Eye family who made me feel at home: The cook, crew on deck, the logistics and medical team who stood stoically intensive hours of GNU Health training. Of course, Selene, the heart of GNU Solidario and the one that looks after the human and non-human family members while I’m away.

You will hardly see these people in the news, because most corporate-backed media neglect them and their organizations. Unlike some billionaire “philanthropists” that take the media spotlight, these anonymous heroes stand on the right side of history, making a difference on the present and future of those who need it most, with very limited resources.

Collage of several pictures during my stay at the Sea-eye

We’re very happy and proud to see that GNU Health can be of help to Sea-Eye in tasks such as guests registration, health evaluations, reporting, statistics and stock management. This is just the beginning and we will be optimizing and adding functionality on successive missions. That said, GNU Health will always play a secondary role compared to picking up somebody from the water and giving them a welcoming hug. Again, we’re a social project with a bit of technology behind.

Drawings made by the children rescued at the Sea-eye

I’d like to finish with a reflection on the picture I took to some of the drawings done by children during their stay at the Sea-Eye. The drawings exist because the Sea-eye crew rescued those kids. Otherwise, their corpses would be at the bottom of the Mediterranean sea, along with thousands who tragically perished trying to find dignity in this world. Thank you, Sea-eye. You are priceless.

A final note: shame on those countries and governments that detain and punish Search and Rescue vessels. Saving lives is not a crime.

Love, freedom and happy hacking

You can obtain Sara Belles painting and Laia Porcar poem from L’Aurora solidarity shop[8]

  1. Laia Porcar : https://laravalerateatre.com/qui-som/
  2. Sara Belles . https://sarabelles.es/
  3. The GNU Health project. https://www.gnuhealth.org
  4. GNU Solidario. Advancing Social Medicine https://www.gnusolidario.org
  5. The GNU Operating System. https://www.gnu.org
  6. Search and rescue on the central Mediterranean migratory route . https://https://www.gnuhealthcon.org/2023/presentations/GHCon2023-Friday-07-Julian_Sassenscheidt-Search_and_rescue_on_the_central_Mediterranean_migratory_route.pdf
  7. The Chaos Computer Club (CCC) . https://www.ccc.de/en/
  8. L’Aurora suport. https://aurorasuport.org/

27 June, 2024 07:48PM by Luis Falcon

Greg Casamento

Free as in Freedom, not as in beer...

 So... recently I was working for a bit (sweat equity or so I thought) for a company by the name of ImmortalData.  The company is headed by a man by the name of Dale Amon.  I have worked, on and off, for them for about 2-3 years.   They are developing a piece of software that is used to extract data from their proprietary black box systems.  This piece of software uses GNUstep.   They were born from a previous company known as XCOR which was developing a space plane at the Mojave space port.   That company is now defunct.

Okay, so with that bit of history, I worked for a while for XCOR and then, because ImmortalData inherited the software, for them as well.  When I worked for XCOR it was as a contractor.  There have been issues with the software (some GNUstep bugs and some bugs due to problems introduced by Dale) that I have been asked to address.

At the end of a meeting a few weeks ago Dale made a comment like "Well, this issue seems like a GNUstep bug, so there is no reason we should have to pay for any of this" which hit an EXTREMELY sour note with me.

Later on that week I tried to clarify it with Dale, and it seems as though he was under the impression that since I was working on Free Software any changes or fixes TO that software should not be billable.   This is NOT true.  Additionally, the issue that they are experiencing is because of something THEY did, and it is not a GNUstep bug. 

I mentioned this in the previous post, but I feel strongly that this needs to be called out explicitly.   Free Software is free as in FREEDOM.  This means you are free to look at, examine, and modify the software as you see fit.   It does NOT mean services performed on that software on your behalf by someone other than you are free.

This development was VERY upsetting to me and I feel the need to make the above VERY clear.

27 June, 2024 10:16AM by Unknown (noreply@blogger.com)

June 26, 2024

FSF News

June 24, 2024

parallel @ Savannah

GNU Parallel 20240622 ('34 counts') released

GNU Parallel 20240622 ('34 counts') has been released. It is available for download at: lbry://@GnuParallel:4

Quote of the month:

  The most glorious 15,000 lines of Perl ever written.
    -- @nibblrrr7124@YouTube
 
New in this release:

  • Bug fixes and man page updates.


News about GNU Parallel:


GNU Parallel - For people who live life in the parallel lane.

If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.


About GNU Parallel


GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
    12345678 883c667e 01eed62f 975ad28b 6d50e22a
    $ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
    cc21b4c9 43fd03e9 3ae1ae49 e28573c0
    $ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
    79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
    fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
  • Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference


If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)


If GNU Parallel saves you money:



About GNU SQL


GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.


About GNU Niceload


GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

24 June, 2024 07:00PM by Ole Tange

GNU Guile

GNU Guile 3.0.10 released

We are pleased to finally announce the release of GNU Guile 3.0.10! This release is mainly a bug-fix release, though it does include a number of new features:

For full details, see the release announcement, and check out the download page.

Happy Guile hacking!

24 June, 2024 08:25AM by Andy Wingo (guile-devel@gnu.org)

June 21, 2024

automake @ Savannah

automake 1.16.92 pretest release candidate

automake 1.16.92 pretest release candidate released. Please test if you can, so 1.17 will be as reliable as we can make it. Announcement:
https://lists.gnu.org/archive/html/autotools-announce/2024-06/msg00001.html

21 June, 2024 10:01PM by Karl Berry

health @ Savannah

MyGNUHealth 2.2 series released!

Dear all

I am happy to announce the release of MyGNUHealth 2.2.0!

The new series of the GNU Health Personal Health record comes with many improvements and bug fixes. Some highlights of this new version:

  • Support for Kivy 2.3.0
  • Localization. MyGNUHealth now has support for different languages. English, Spanish and Chinese are available to use, and French, German, Italian are ready to be translated. There will be a translation component for MyGNUHealth at Codeberg's Weblate instance.
  • Bluetooth functionality: Starting with MyGH series 2.2 we provide bluetooth integration for open compatible devices and health trackers. We include the link with the Pinetime Smartwatch (experimental) and the possibility to link to any open hardware device (glucometer, scales, blood pressure monitors,  .. ). We need to get a list of available medical devices that respect our privacy and freedom, so let us know of any!
  • Charts now allow to select date ranges with calendar widgets
  • The Book of Life have a revised format for the pages.
  • The charts have been improved in the format and include x axis labels.


Thanks to Kivy, Mygnuhealth codebase can be ported to other architectures and operating systems such as Android AOSP (Pierre Michel is working on this) and GNU/Linux phones.

In addition to Savannah, we have incorporated Codeberg to the GNU Health development environment. Mailing lists, news and file downloads are at GNU, while the development repositories are at Codeberg (https://codeberg.org/gnuhealth)

You can download the latest MyGNUhealth sourcecode from GNU ftp site, pypi (using pip) or from your operating system package (like openSUSE).

Upgrading should be straightforward, and all the health history will remain in the MyGH database. In any case, please make sure you make a backup before upgrading (and daily ;) ).

Thank you to all the contributors that have possible this milestone!

Happy hacking
Luis

21 June, 2024 09:44AM by Luis Falcon

June 17, 2024

Greg Casamento

Keysight laid me off in January!

A little history first. Keysight is a large company that, primarily, makes testing equipment such as oscilloscopes and other electronics. They bought a company a few years back named TestPlant. Prior to that, TestPlant bought a company by the name of Redstone that produced a product known as Eggplant. Recently, I was laid off for economic reasons (at least that's what they said). It occurs to me that nothing in this world lasts forever. I was so depressed when I was let go because Keysight was the perfect home for me... they used GNUstep deeply. So, as you can imagine, I was deeply upset when things ended... but all things do. 

 I think it happened for several reasons: 
  • Economic - This is what was explained to me, but I am not sure I believe it 
  • Politics - I think this part is because I expressed my opinions HONESTLY about the direction of the company given that they wanted to make the application into a VSCode plugin.
  • Perception - I am 54 years old... so I think that they believed that Objective-C was my one and only talent, it's not... I know many other languages and have many other skills. 
Unfortunately, in the US, any employer can let go of any employee or contractor for ANY reason. This is known as at-will employment, making it very hard to take any action against any employer (not that this is something I considered).

Keysight is and will remain a major contributor to GNUstep.

That being said, I recently ran into something rather disturbing at another company.   I have been working with a company based out of New Mexico that is interested in space applications.  They have been using GNUstep and have been awaiting funding.

The lead of this effort expressed something during a meeting saying "We will work on the GNUstep side of this because there is no reason we should have to pay for any of this."   This hit a sour note with me to say the very least.   As it turns out he was under the mistaken impression that, because the work was on GNUstep, it was for free... which is WRONG.

I wonder if the same impression was present at Keysight or if other companies believe this.  The saying, according to RMS, is "Free as in freedom, not as in beer."   If you are a manager at a company who is under the mistaken impression that work on any Free Software or Open Source project is free when your product depends on it, please correct your thinking.   Just because it is someone's passion project does NOT mean that they are going to do that work for free and prioritize the things that need to be done for your organization.

All of that being said the positive sides are this:
  1. More time to code on GNUstep without interruption
  2. More time to work on my own projects
  3. Time to rest and relax
So, as much as I hate being unemployed there ARE some upsides to it.  Here's to hoping something works out soon.   I literally loved my job at Keysight and, honestly, hope to return.   I have my eye on their changes as well as those of others just like any other member of the community.  Yours, GC

17 June, 2024 05:24AM by Unknown (noreply@blogger.com)

June 07, 2024

GNUnet News

GNUnet 0.21.2

GNUnet 0.21.2

This is a bugfix release for gnunet 0.21.1. It primarily addresses some connectivity issues introduced with our new transport subsystem.

Links

The GPG key used to sign is: 3D11063C10F98D14BD24D1470B0998EF86F59B6A

Note that due to mirror synchronization, not all links may be functional early after the release. For direct access try https://ftp.gnu.org/gnu/gnunet/

libgnunetchat 0.5.0 released

We are also pleased to announce the release of libgnunetchat 0.5.0.
This is a major new release bringing compatibility with the major changes in the Messenger service from latest GNUnet release 0.21.2 adding new message kinds and functionality. This release will also require your GNUnet to be at least 0.21.2 because of that.

Download links

Noteworthy changes in 0.5.0

  • This release requires the GNUnet Messenger Service 0.5!
  • Implements tickets to share attributes with contacts.
  • Implement functionality to get recipient of sent private messages.
  • Allow file sharing without additional encryption key.
  • Implements discourses to send data in live channels.
  • Fix memory violations and duplicate storage entries.
  • Adjust callbacks regarding account states.
  • Fix deletions of accounts and lobbies.
  • Fix multiple synchronization bugs.
  • Add test cases for discourses and tickets.

A detailed list of changes can be found in the ChangeLog .

Messenger-GTK 0.10.0

Since libgnunetchat made some changes there're also a new releases of the messenger applications addressing changes for compatibility and providing some new functionality.

Download links

Noteworthy changes in 0.10.0

  • Implement tagging and filtering messages
  • Adjust media previews and optimize memory footprint
  • Implement sharing profile attribute and profile picture with contacts
  • Fix several UI issues and memory leaks
  • Improve UI to waste less vertical space for smaller screens
  • Add localization for English, German and Spanish

Known Issues

  • Chats still require a reliable connection between GNUnet peers. So this still depends on the upcoming NAT traversal to be used outside of local networks for most users (see #5710 ).
  • File sharing via the FS service should work in a GNUnet single-user setup but a multi-user setup breaks it (see #7355 )

In addition to this list, you may also want to consult our bug tracker at bugs.gnunet.org .

messenger-cli 0.3.0

This is mostly a compatibility release for messenger-cli 0.3.0 to address changes in libgnunetchat 0.5.0.

Download links

07 June, 2024 10:00PM

www-zh-cn @ Savannah

copyright notices in www.gnu.org translations

Dear Translators:

Recently, the Licensing and Compliance Lab provided guidelines
for writing copyright notices in www.gnu.org translations:

https://www.gnu.org/s/trans-coord/w/Copyright-Notices.html

Please take them into account.

After received 2 translators‘ feedback plus my thought, I would put the following as advice for new translations:

1. add your name in the copyright notices in the translation if you think your contribution is enough for an article, like

Copyright &copy; 2024 Free Software Foundation, Inc.<br></br>
Copyright &copy; 2024 XIE Wensheng (translation)<

2. or optionally add your name in the TRANSLATOR'S CREDITS part as we always do.

<b>翻译</b>:李凡希,2010。<br></br>
<b>审校</b>:<a href="mailto:1945649519@qq.com">&lt;Nios34&gt;</a>,2020。<br></br>
<b>翻译团队</b>:<a rel="team" href="https://savannah.gnu.org/projects/www-zh-cn/">&lt;CTT&gt;</a>,2017-2024。<

best regards,
wxie

07 June, 2024 10:15AM by Wensheng XIE

gsl @ Savannah

GNU Scientific Library 2.8 released

Version 2.8 of the GNU Scientific Library (GSL) has been released.
Thank you to all who helped test the library prior to the release, and
thank you to everyone for using the library and giving feedback and
reports. The following changes have been added to the library:

  • What is new in gsl-2.8:


** apply patch for bug #63679 (F. Weimer)

** updated multilarge TSQR method to store ||z_2|| and
   provide it to the user

** add routines for Hermite B-spline interpolation

** fix for bug #59624

** fix for bug #59781 (M. Dunlap)

** bug fix #61094 (reported by A. Cheylus)

** add functions:
   - gsl_matrix_complex_conjugate
   - gsl_vector_complex_conj_memcpy
   - gsl_vector_complex_div_real
   - gsl_linalg_QR_lssolvem_r
   - gsl_linalg_complex_QR_lssolvem_r
   - gsl_linalg_complex_QR_QHmat_r
   - gsl_linalg_QR_UR_lssolve
   - gsl_linalg_QR_UR_lssvx
   - gsl_linalg_QR_UR_QTvec
   - gsl_linalg_QR_UU_lssvx
   - gsl_linalg_QR_UD_lssvx
   - gsl_linalg_QR_UD_QTvec
   - gsl_linalg_complex_cholesky_{decomp2,svx2,solve2,scale,scale_apply}
   - gsl_linalg_SV_{solve2,lssolve}
   - gsl_rstat_norm

** add Lebedev quadrature (gsl_integration_lebedev)

** major overhaul to the B-spline module to add
   new functionality

07 June, 2024 01:10AM by Patrick Alken

June 05, 2024

enscript @ Savannah

GNU Enscript 1.7rc released

Version 1.7rc is available for download from:

  git clone https://git.savannah.gnu.org/git/enscript.git

We are looking forward for your feedback.

05 June, 2024 12:21PM by Wim Stockman

June 04, 2024

FSF Events

Free Software Directory meeting on IRC: Friday, June 07, starting at 12:00 EDT (16:00 UTC)

Join the FSF and friends on Friday, June 07, from 12:00 to 15:00 EDT (16:00 to 19:00 UTC) to help improve the Free Software Directory.

04 June, 2024 08:28PM

June 03, 2024

FSF News

June 01, 2024

findutils @ Savannah

GNU findutils 4.10.0 released

This is to announce findutils-4.10.0, a stable release.
See the NEWS below for more details.

GNU findutils is a set of software tools for finding files that match
certain criteria and for performing various operations on them.
Findutils includes the programs "find", "xargs" and "locate".
More information about findutils is available at:
  https://www.gnu.org/software/findutils/

Please report bugs and problems with this release via the the
GNU Savannah bug tracker:
  https://savannah.gnu.org/bugs/?group=findutils

Please send general comments and feedback about the GNU findutils
package to the mailing list (<mailto:bug-findutils@gnu.org):
  https://lists.gnu.org/mailman/listinfo/bug-findutils

There have been 88 commits by 8 people in the - sigh - 121 weeks since 4.9.0:
  Antonio Diaz Diaz (2)       James Youngman (24)
  Bernhard Voelker (57)       John A. Leuenhagen (1)
  Bjarni Ingi Gislason (1)    Shuiqing Zhou (1)
  Helmut Grohne (1)           ribbon (1)

This release was bootstrapped with the following tools:
   Autoconf 2.72
   Automake 1.16.5
   M4 1.4.18
   Gnulib v1.0-187-g623bcc22f4

Please consider supporting the Free Software Foundation in its fund
raising appeal; see <https://www.fsf.org/appeal/>.

Thanks to everyone who has contributed!

Have a nice day,
Bernhard Voelker [on behalf of the GNU findutils maintainers]

================================================================================

Here are the compressed sources:
  https://ftp.gnu.org/pub/gnu/findutils/findutils-4.10.0.tar.xz
   
Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/pub/gnu/findutils/findutils-4.10.0.tar.xz.sig

Use a mirror for higher download bandwidth:
  http://www.gnu.org/order/ftp.html

Here is the SHA256 checksum:
 
  1387e0b67ff247d2abde998f90dfbf70c1491391a59ddfecb8ae698789f0a4f5  findutils-4.10.0.tar.xz

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

gpg --verify findutils-4.10.0.tar.xz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

gpg --keyserver keys.gnupg.net --recv-keys A5189DB69C1164D33002936646502EF796917195

and rerun the 'gpg --verify' command.

================================================================================

NEWS

  • Noteworthy changes in release 4.10.0 (2024-06-01) [stable]


** Bug Fixes

  Find now defaults to optimization level 1 rather than 2 and the
  cost-based optimizer will only run at level 2 and above.  This
  should prevent changes of operation order which result in
  user-visible differences in behaviour. [#58427]

  If the -P option to xargs is not used, xargs will not change the way
  in which the SIGUSR1 and SIGUSR2 signals are handled.  This means
  that they will cause the program to terminate if the signals were
  not ignored in the process which started xargs.  This also means that
  xargs does not use parallel execution at all.
  If you start xargs with '-P 1', then xargs will not be killed by these
  signals, and they instead change the degree of parallelism.
  This change improves xargs' POSIX compliance.

  'xargs -P' now waits for all its child processes to complete before
  exiting, even if one of them exits with status 255. [#64451]

  If the -P option of xargs is in use, reads on standard input which are
  interrupted by a signal are re-started. [#64442]

  'find -name /' no longer outputs a warning, because that is a valid pattern
  to match the root directory "/".  Previously, a diagnostic falsely claimed
  that this pattern would not match anything. [#62227]

  'find -gid' (without the mandatory argument) now outputs a correct error
  diagnostic.  Previously it output: "find: invalid argument `-gid' to `-gid'".
  The error diagnostic for non-numeric arguments has been improved as well.
  Likewise for -inum, -links and -uid.

  'find -user' and 'find -group' now allow to specify larger UIDs/GIDs.
  Previously, that was limited to INT_MAX, although the types uid_t and gid_t
  are larger on many systems, including x86_64 GNU/Linux. [#64900]

  'find -xtype l' no longer fails on symbolic links that point to
  themselves.  These are treated similarly to broken links. [#51926]

** Improvements

  The find predicates -used, -amin, -cmin, -mmin, -atime, -ctime, and -mtime
  now properly diagnose a not-a-number argument.  Previously, find dumped
  core via an assertion.  [#64717]

** Changes to the build process

  findutils now builds again on systems with musl-libc.
  This requires gettext-0.19.8.

  findutils programs no longer fail for timestamps past the year 2038
  on obsolete configurations with 32-bit signed time_t, because the
  build procedure now rejects these configurations.
  On systems without any year2038 support configure with --disable-year2038.

** Documentation Changes

  When generating the Texinfo manual, `makeinfo` is invoked with the --no-split
  option for all output formats now; this avoids files like find.info-[12].

  The xargs documentation now describes the double dash "--" option delimiter.

  The xargs examples in the Texinfo manual now use the -L and --replace options
  instead of the deprecated -l and -i options.  [#64480]

  The TexInfo manual now uses upper-case 'B' as birthtime for the -newerXY
  comparison consistently.  [#65378]

** Translations

Updated the following translations: Belarusian, Brazilian Portuguese,
Bulgarian, Catalan, Chinese (simplified), Chinese (traditional),
Croatian, Czech, Danish, Dutch, Esperanto, Estonian, Finnish, French,
Galician, Georgian, German, Greek, Hungarian, Indonesian, Irish,
Italian, Japanese, Korean, Lithuanian, Luganda, Malay, Norwegian
Bokmaal, Polish, Portuguese, Romanian, Russian, Serbian, Slovak,
Slovenian, Spanish, Swedish, Turkish, Ukrainian, Vietnamese.

01 June, 2024 06:30PM by Bernhard Voelker

May 31, 2024

poke @ Savannah

GNU poke 4.1 released

I am happy to announce a new release of GNU poke, version 4.1.

This is a bugfix release in the 4.x series.

See the file NEWS in the distribution tarball for a list of issues
fixed in this release.

The tarball poke-4.1.tar.gz is now available at
https://ftp.gnu.org/gnu/poke/poke-4.1.tar.gz.

> GNU poke (http://www.jemarch.net/poke) is an interactive, extensible
> editor for binary data.  Not limited to editing basic entities such
> as bits and bytes, it provides a full-fledged procedural,
> interactive programming language designed to describe data
> structures and to operate on them.


Thanks to the people who contributed with code and/or documentation to
this release.

Happy poking!

Mohammad-Reza Nabipoor

31 May, 2024 02:32PM by Mohammad-Reza Nabipoor

GNU Guix

Source code archiving in Guix: new publication

We are glad to announce the publication of a new research paper entitled Source Code Archiving to the Rescue of Reproducible Deployment for the ACM Conference on Reproducibility and Replicability. The paper presents work that has been done since we started connecting Guix with the Software Heritage (SWH) archive five years ago:

The ability to verify research results and to experiment with methodologies are core tenets of science. As research results are increasingly the outcome of computational processes, software plays a central role. GNU Guix is a software deployment tool that supports reproducible software deployment, making it a foundation for computational research workflows. To achieve reproducibility, we must first ensure the source code of software packages Guix deploys remains available.

We describe our work connecting Guix with Software Heritage, the universal source code archive, making Guix the first free software distribution and tool backed by a stable archive. Our contribution is twofold: we explain the rationale and present the design and implementation we came up with; second, we report on the archival coverage for package source code with data collected over five years and discuss remaining challenges.

The ability to retrieve package source code is important for researchers who need to be able to replay scientific workflows, but it’s just as important for engineers and developers alike, who may also have good reasons to redeploy or to audit past package sets.

Support for source code archiving and recovery in Guix has improved a lot over the past five years, in particular with:

  • Support for recovering source code tarballs (tar.gz and similar files): this is made possible by Disarchive, written by Timothy Sample.

Diagram taken from the paper showing Disarchive tarball “disassembly” and “assembly”.

  • The ability to look up data by nar hash in the SWH archive (“nar” is the normalized archive format used by Nix and Guix), thanks to fellow SWH hackers. This, in turn, allows Guix to look up any version control checkout by content hash—Git, Subversion, Mercurial, you name it!
  • The monitoring of archival coverage with Timothy’s Preservation of Guix reports has allowed us to identify discrepancies in Guix, Disarchive, and/or SWH and to increase archival coverage.

Graph taken from the paper showing package source code archival coverage over time.

94% of the packages in a January 2024 snapshot of Guix are known to have their source code archived!

Check out the paper to learn more about the machinery at play and the current status.

31 May, 2024 12:00PM by Ludovic Courtès, Timothy Sample, Simon Tournier, Stefano Zacchiroli

May 28, 2024

FSF Events

May 24, 2024

Parabola GNU/Linux-libre

pacman.conf and makepkg.conf changes

NOTICE FOR EVERYONE:

You may see the following error message from pacman:

error: config file /etc/pacman.d/*.conf could not be read: No such file or directory

If you do, that is because you have modified your pacman.conf file in the past; but you forgot to reconcile the latest .pacnew replacement. Remember that it is upon each user to notice any new .pacnew replacement files for any configuration files that you have modified, each time pacman does an upgrade, and to merge the changes into your existing config files. Pacman will not do that automatically, to avoid clobbering your customizations. To correct this, compare /etc/pacman.conf and /etc/pacman.conf.pacnew to remind yourself which changes you had made. Then move /etc/pacman.conf.pacnew to /etc/pacman.conf; and redo you customizations (eg: enable the [nonsystemd] repo, enable ParallelDownloads, add ILoveCandy, etc).

NOTICE FOR THOSE WHO BUILD THEIR OWN PACKAGES:

Parabola's default makepkg.conf has long loaded /etc/makepkg.d/*.conf. As of makepkg 6.1.0, the program itself now loads /etc/makepkg.conf.d/*.conf, so this part of our makepkg.conf has been removed. Users who have /etc/makepkg.d/*.conf files need to move them to /etc/makepkg.conf.d/.

24 May, 2024 01:06AM by Luke Shumaker

May 22, 2024

FSF Events

Free Software Directory meeting on IRC: Friday, May 24, starting at 12:00 EDT (16:00 UTC)

Join the FSF and friends on Friday, May 24, from 12:00 to 15:00 EDT (16:00 to 19:00 UTC) to help improve the Free Software Directory.

22 May, 2024 10:32PM

parallel @ Savannah

GNU Parallel 20240522 ('Tbilisi') released

GNU Parallel 20240522 ('Tbilisi') has been released. It is available for download at: lbry://@GnuParallel:4

Quote of the month:

  GNU Parallel é mais um daqueles "como eu vivia sem isso?!"
  -- Ivan Augusto @ivanaugustobd@twitter
 
New in this release:

  • --onall now supports sshpass - user:pass@host.
  • --memfree kills do not count as --retries.
  • Bug fixes and man page updates.


GNU Parallel - For people who live life in the parallel lane.

If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.


About GNU Parallel


GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
    12345678 883c667e 01eed62f 975ad28b 6d50e22a
    $ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
    cc21b4c9 43fd03e9 3ae1ae49 e28573c0
    $ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
    79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
    fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
  • Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference


If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)


If GNU Parallel saves you money:



About GNU SQL


GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.


About GNU Niceload


GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

22 May, 2024 08:43PM by Ole Tange

May 18, 2024

education @ Savannah

May 13, 2024

libtool @ Savannah

libtool-2.5.0 released [alpha]

Libtoolers!

The Libtool Team is pleased to announce the release of libtool 2.5.0, a alpha release.

GNU Libtool hides the complexity of using shared libraries behind a
consistent, portable interface. GNU Libtool ships with GNU libltdl, which
hides the complexity of loading dynamic runtime libraries (modules)
behind a consistent, portable interface.

There have been 91 commits by 29 people in the 113 weeks since 2.4.7.

See the NEWS below for a brief summary.

Thanks to everyone who has contributed!
The following people contributed changes to this release:

  Albert Chu (1)
  Alex Ameen (3)
  Antonin Décimo (3)
  Brad Smith (2)
  Bruno Haible (2)
  Dmitry Antipov (1)
  Florian Weimer (1)
  Gilles Gouaillardet (1)
  Ileana Dumitrescu (24)
  Jakub Wilk (1)
  Jonathan Wakely (2)
  Manoj Gupta (1)
  Mike Frysinger (23)
  Mingli Yu (2)
  Oliver Kiddle (1)
  Olly Betts (1)
  Ozkan Sezer (2)
  Paul Eggert (2)
  Paul Green (1)
  Raul E Rangel (1)
  Richard Purdie (5)
  Sam James (4)
  Samuel Thibault (1)
  Stephen Webb (1)
  Tijl Coosemans (1)
  Tim Rice (1)
  Uwe Kleine-König (1)
  Vadim Zeitlin (1)
  Xiang.Lin (1)

Ileana
 [on behalf of the libtool maintainers]
==================================================================

Here is the GNU libtool home page:
    https://gnu.org/s/libtool/

For a summary of changes and contributors, see:
  https://git.sv.gnu.org/gitweb/?p=libtool.git;a=shortlog;h=v2.5.0
or run this command from a git-cloned libtool directory:
  git shortlog v2.4.7..v2.5.0

Here are the compressed sources:
  https://alpha.gnu.org/gnu/libtool/libtool-2.5.0.tar.gz   (1.9MB)
  https://alpha.gnu.org/gnu/libtool/libtool-2.5.0.tar.xz   (1008KB)

Here are the GPG detached signatures:
  https://alpha.gnu.org/gnu/libtool/libtool-2.5.0.tar.gz.sig
  https://alpha.gnu.org/gnu/libtool/libtool-2.5.0.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

  fb3ab5907115b16bf12a0d3d424c79cb0003d02e  libtool-2.5.0.tar.gz
  1DjDF0VdhVVM4vmYvkiGb9QM/L+DTWCzAm9PwO1YPSM=  libtool-2.5.0.tar.gz
  70e2dd113a9460c279df01b2eee319adb99ee998  libtool-2.5.0.tar.xz
  fhDMhjgj1AjsX/6kHUPDckqgiBZldXljydsL77LIecw=  libtool-2.5.0.tar.xz

Verify the base64 SHA256 checksum with cksum -a sha256 --check
from coreutils-9.2 or OpenBSD's cksum since 2007.

Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify libtool-2.5.0.tar.gz.sig

The signature should match the fingerprint of the following key:

  pub   rsa4096 2021-09-23 [SC]
        FA26 CA78 4BE1 8892 7F22  B99F 6570 EA01 146F 7354
  uid   Ileana Dumitrescu <ileanadumi95@protonmail.com>
  uid   Ileana Dumitrescu <ileanadumitrescu95@gmail.com>

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --locate-external-key ileanadumi95@protonmail.com

  gpg --recv-keys 6570EA01146F7354

  wget -q -O- 'https://savannah.gnu.org/project/release-gpgkeys.php?group=libtool&download=1' | gpg --import -

As a last resort to find the key, you can try the official GNU
keyring:

  wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
  gpg --keyring gnu-keyring.gpg --verify libtool-2.5.0.tar.gz.sig

This release was bootstrapped with the following tools:
  Autoconf 2.72e
  Automake 1.16.5
  Gnulib v0.1-6995-g29d705ead1

NEWS

  • Noteworthy changes in release 2.5.0 (2024-05-13) [alpha]


** New features:

  - Pass '-fdiagnostics-color', '-frecord-gcc-switches',
    '-fno-sanitize*', '-Werror', and 'prefix-map' flags.

  - Pass the '-no-canonical-prefixes' linker flag.

  - Pass '-fopenmp=*' for Clang to allow choosing between libgomp and
    libomp.

  - Pass '-shared-libsan', '-static-libsan', 'rtlib=*', and
    'unwindlib=*' for Clang.

  - Expanded process.h inclusion on Windows for more than the
    proprietary MSVC compiler. Other alternative Windows compilers
    also require process.h.

  - Pass 'elf32_x86_64' and 'elf64_x86_64' to the linker on hurd-amd64.

  - Recognize --windows* config triplets.

** Important incompatible changes:

  - Removed test_compile from command line options.

  - By default executables are created with the RUNPATH property for
    the Android linker. RUNPATH works for libraries which are not
    installed in system locations.

  - Removed AC_PROG_SED fallback, as the macro has been supported
    in Autoconf since the 90's.

** Bug fixes:

  - Check for space after -l, -L, and -R linker flags.

  - Updated documentation for tests, the demo directory, and
    elsewhere.

  - Fixed Solaris 11 builds.

  - Clean trailing "/" from sysroot path.

  - Fixed shared library builds for System V.

  - Added mingw to the list of systems not requiring libm.

  - Fixed support for nios2 systems.

  - Fixed linker check for '--whole-archive' support for linkers other
    than ld.

  - Use -Fe instead of -o with MSVC to avoid deprecation warnings.

  - Improved reproducibility of libtool scripts.

  - Avoided MinGW warning by adding CRTIMP.

  - Improved grep portability.

  - Fixed cross-building warnings when checking for file.


** Changes in supported systems or compilers:

  - Removed support for bitrig (--bitrig*).

  - Added support for flang (Fortran LLVM-based) compilers.


Enjoy!

13 May, 2024 07:06PM by Ileana Dumitrescu

May 07, 2024

GNU Guix

Authenticate your Git checkouts!

You clone a Git repository, then pull from it. How can you tell its contents are “authentic”—i.e., coming from the “genuine” project you think you’re pulling from, written by the fine human beings you’ve been working with? With commit signatures and “verified” badges ✅ flourishing, you’d think this has long been solved—but nope!

Four years after Guix deployed its own tool to allow users to authenticate updates fetched with guix pull (which uses Git under the hood), the situation hasn’t changed all that much: the vast majority of developers using Git simply do not authenticate the code they pull. That’s pretty bad. It’s the modern-day equivalent of sharing unsigned tarballs and packages like we’d blissfully do in the past century.

The authentication mechanism Guix uses for channels is available to any Git user through the guix git authenticate command. This post is a guide for Git users who are not necessarily Guix users but are interested in using this command for their own repositories. Before looking into the command-line interface and how we improved it to make it more convenient, let’s dispel any misunderstandings or misconceptions.

Why you should care

When you run git pull, you’re fetching a bunch of commits from a server. If it’s over HTTPS, you’re authenticating the server itself, which is nice, but that does not tell you who the code actually comes from—the server might be compromised and an attacker pushed code to the repository. Not helpful. At all.

But hey, maybe you think you’re good because everyone on your project is signing commits and tags, and because you’re disciplined, you routinely run git log --show-signature and check those “Good signature” GPG messages. Maybe you even have those fancy “✅ verified” badges as found on GitLab and on GitHub.

Signing commits is part of the solution, but it’s not enough to authenticate a set of commits that you pull; all it shows is that, well, those commits are signed. Badges aren’t much better: the presence of a “verified” badge only shows that the commit is signed by the OpenPGP key currently registered for the corresponding GitLab/GitHub account. It’s another source of lock-in and makes the hosting platform a trusted third-party. Worse, there’s no notion of authorization (which keys are authorized), let alone tracking of the history of authorization changes (which keys were authorized at the time a given commit was made). Not helpful either.

Being able to ensure that when you run git pull, you’re getting code that genuinely comes from authorized developers of the project is basic security hygiene. Obviously it cannot protect against efforts to infiltrate a project to eventually get commit access and insert malicious code—the kind of multi-year plot that led to the xz backdoor—but if you don’t even protect against unauthorized commits, then all bets are off.

Authentication is something we naturally expect from apt update, pip, guix pull, and similar tools; why not treat git pull to the same standard?

Initial setup

The guix git authenticate command authenticates Git checkouts, unsurprisingly. It’s currently part of Guix because that’s where it was brought to life, but it can be used on any Git repository. This section focuses on how to use it; you can learn about the motivation, its design, and its implementation in the 2020 blog post, in the 2022 peer-reviewed academic paper entitled Building a Secure Software Supply Chain with GNU Guix, or in this 20mn presentation.

To support authentication of your repository with guix git authenticate, you need to follow these steps:

  1. Enable commit signing on your repo: git config commit.gpgSign true. (Git now supports other signing methods but here we need OpenPGP signatures.)

  2. Create a keyring branch containing all the OpenPGP keys of all the committers, along these lines:

    git checkout --orphan keyring
    git reset --hard
    gpg --export alice@example.org > alice.key
    gpg --export bob@example.org > bob.key
    …
    git add *.key
    git commit -m "Add committer keys."

    All the files must end in .key. You must never remove keys from that branch: keys of users who left the project are necessary to authenticate past commits.

  3. Back to the main branch, add a .guix-authorizations file, listing the OpenPGP keys of authorized committers—we’ll get back to its format below.

  4. Commit! This becomes the introductory commit from which authentication can proceed. The introduction of your repository is the ID of this commit and the OpenPGP fingerprint of the key used to sign it.

That’s it. From now on, anyone who clones the repository can authenticate it. The first time, run:

guix git authenticate COMMIT SIGNER

… where COMMIT is the commit ID of the introductory commit, and SIGNER is the OpenPGP fingerprint of the key used to sign that commit (make sure to enclose it in double quotes if there are spaces!). As a repo maintainer, you must advertise this introductory commit ID and fingerprint on a web page or in a README file so others know what to pass to guix git authenticate.

The commit and signer are now recorded on the first run in .git/config; next time, you can run it without any arguments:

guix git authenticate

The other new feature is that the first time you run it, the command installs pre-push and pre-merge hooks (unless preexisting hooks are found) such that your repository is automatically authenticated from there on every time you run git pull or git push.

guix git authenticate exits with a non-zero code and an error message when it stumbles upon a commit that lacks a signature, that is signed by a key not in the keyring branch, or that is signed by a key not listed in .guix-authorizations.

Maintaining the list of authorized committers

The .guix-authorizations file in the repository is central: it lists the OpenPGP fingerprints of authorized committers. Any commit that is not signed by a key listed in the .guix-authorizations file of its parent commit(s) is considered inauthentic—and an error is reported. The format of .guix-authorizations is based on S-expressions and looks like this:

;; Example ‘.guix-authorizations’ file.

(authorizations
 (version 0)               ;current file format version

 (("AD17 A21E F8AE D8F1 CC02  DBD9 F8AE D8F1 765C 61E3"
   (name "alice"))
  ("2A39 3FFF 68F4 EF7A 3D29  12AF 68F4 EF7A 22FB B2D5"
   (name "bob"))
  ("CABB A931 C0FF EEC6 900D  0CFB 090B 1199 3D9A EBB5"
   (name "charlie"))))

The name bits are hints and do not have any effect; what matters is the fingerprints that are listed. You can obtain them with GnuPG by running commands like:

gpg --fingerprint charlie@example.org

At any time you can add or remove keys from .guix-authorizations and commit the changes; those changes take effect for child commits. For example, if we add Billie’s fingerprint to the file in commit A, then Billie becomes an authorized committer in descendants of commit A (we must make sure to add Billie’s key as a file in the keyring branch, too, as we saw above); Billie is still unauthorized in branches that lack A. If we remove Charlie’s key from the file in commit B, then Charlie is no longer an authorized committer, except in branches that start before B. This should feel rather natural.

That’s pretty much all you need to know to get started! Check the manual for more info.

All the information needed to authenticate the repository is contained in the repository itself—it does not depend on a forge or key server. That’s a good property to allow anyone to authenticate it, to ensure determinism and transparency, and to avoid lock-in.

Interested? You can help!

guix git authenticate is a great tool that you can start using today so you and fellow co-workers can be sure you’re getting the right code! It solves an important problem that, to my knowledge, hasn’t really been addressed by any other tool.

Maybe you’re interested but don’t feel like installing Guix “just” for this tool. Maybe you’re not into Scheme and Lisp and would rather use a tool written in your favorite language. Or maybe you think—and rightfully so—that such a tool ought to be part of Git proper.

That’s OK, we can talk! We’re open to discussing with folks who’d like to come up with alternative implementations—check out the articles mentioned above if you’d like to take that route. And we’re open to contributing to a standardization effort. Let’s get in touch!

Acknowledgments

Thanks to Florian Pelz and Simon Tournier for their insightful comments on an earlier draft of this post.

07 May, 2024 02:14PM by Ludovic Courtès

May 05, 2024

FSF News

May 02, 2024

Gary Benson

git submodule forgetting

Did you forget the -r when cloning a git repo with submodules? The command you’re looking for is git submodule update --init

02 May, 2024 03:11PM by gbenson

April 29, 2024

FSF News

FSF to be deposed in SFC v Vizio, updates relevant FAQ entry

29 April, 2024 06:23PM

April 26, 2024

gnulib @ Savannah

GNU gnulib: gnulib-tool has become much faster

If you are developer on a package that uses GNU gnulib as part of its build system:

gnulib-tool has been known for being slow for many years. We have listened to your complaints. We have rewritten gnulib-tool in another programming language (Python). It is between 8 times and 100 times faster than the previous implementation.

Both implementations behave identically, that is, produce the same generated files and the same output. Nothing changes in your way to use Gnulib; it's only faster.

In order to reap the new speed:

1. Make sure you have Python (version 3.7 or newer) installed on your machine.

2. Update your gnulib checkout. (For some packages, it comes as a git submodule named 'gnulib'.) Like this:

  $ git checkout master
  $ git pull

  Set the environment variable GNULIB_SRCDIR, pointing to this checkout.

  If the package is using a git submodule named 'gnulib', it is also advisable to do

  $ git commit -m 'build: Update gnulib submodule to latest.' gnulib

  (as a preparation for step 4, because the --no-git option does not work as expected in all variants of 'bootstrap').

3. Clean the built files of your package:

  $ make -k distclean


4. Regenerate the fetched and generated files of your package. Depending on the package, this may be a command such as

  $ ./bootstrap --no-git --gnulib-srcdir=$GNULIB_SRCDIR

  or

  $ export GNULIB_SRCDIR; ./autopull.sh; ./autogen.sh

  or, if no such script is available:

  $ $GNULIB_SRCDIR/gnulib-tool --update


5. Continue with

  $ ./configure
  $ make

  as usual.

Enjoy! The rewritten gnulib-tool was implemented by Dmitry Selyutin, Collin Funk, and me.

26 April, 2024 10:12AM by Bruno Haible

April 22, 2024

parallel @ Savannah

GNU Parallel 20240422 ('Børsen') [stable]

GNU Parallel 20240422 ('Børsen') has been released. It is available for download at: lbry://@GnuParallel:4

Quote of the month:

  I’m a big fan of GNU parallel!
    -- Scott Cain @scottjcain@twitter
 
New in this release:

  • Bug fixes and man page updates.


GNU Parallel - For people who live life in the parallel lane.

If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.


About GNU Parallel


GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
    12345678 883c667e 01eed62f 975ad28b 6d50e22a
    $ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
    cc21b4c9 43fd03e9 3ae1ae49 e28573c0
    $ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
    79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
    fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
  • Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference


If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)


If GNU Parallel saves you money:



About GNU SQL


GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.


About GNU Niceload


GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

22 April, 2024 09:12PM by Ole Tange

www-zh-cn @ Savannah

Welcome our new member - integral

Hi, All:

Please join me in welcoming our new member:

 User Details:
-------------
Name:
Login:   integral
Email:   integral@member.fsf.org

I wish integral a wonderful journey in GNU CTT.

Happy Hacking
wxie

22 April, 2024 12:56AM by Wensheng XIE

April 21, 2024

gnulib @ Savannah

GNU gnulib: calling for beta-testers

If you are developer on a package that uses GNU gnulib as part of its build system:

gnulib-tool has been known for being slow for many years. We have listened to your complaints. A rewrite of gnulib-tool in another programming language (Python) is ready for beta-testing. It is between 8 times and 100 times faster than the original gnulib-tool.

Both implementations should behave identically, that is, produce the same generated files and the same output. You can help us ensure this, through the following steps:

1. Make sure you have Python (version 3.7 or newer) installed on your machine.

2. Update your gnulib checkout. (For some packages, it comes as a git submodule named 'gnulib'.) Like this:

  $ git checkout master
  $ git pull

     Set the environment variable GNULIB_SRCDIR, pointing to this checkout.

     If the package is using a git submodule named 'gnulib', it is also advisable to do

  $ git commit -m 'build: Update gnulib submodule to latest.' gnulib

     (as a preparation for step 5, because the --no-git option does not work as expected in all variants of 'bootstrap').

3. Set an environment variable that enables checking that the two implementations behave the same:

  $ export GNULIB_TOOL_IMPL=sh+py


4. Clean the built files of your package:

  $ make -k distclean


5. Regenerate the fetched and generated files of your package. Depending on the package, this may be a command such as

  $ ./bootstrap --no-git --gnulib-srcdir=$GNULIB_SRCDIR

     or

  $ export GNULIB_SRCDIR; ./autopull.sh; ./autogen.sh

     or, if no such script is available:

  $ $GNULIB_SRCDIR/gnulib-tool --update

     If there is a failure, due to differences between the 'sh' and 'py' results, please report it to <bug-gnulib@gnu.org>.

6. If this invocation was successful, you can trust the rewritten gnulib-tool and use it from now on, by setting the environment variable

  $ export GNULIB_TOOL_IMPL=py


7. Continue with

  $ ./configure
  $ make

     as usual.

And enjoy the speed! The rewritten gnulib-tool was implemented by Dmitry Selyutin, Collin Funk, and me.

21 April, 2024 10:47AM by Bruno Haible

April 17, 2024

www-zh-cn @ Savannah

It is easy to contribute to GNU

I will be delivering my talk, "It is easy to contribute to GNU," Saturday, May 4, 2024, 12:15--13:00 EDT (16:00 UTC), at the LibrePlanet 2024 conference, and I hope you’ll check it out!

LibrePlanet is a conference about software freedom, happening on May 4 & 5, 2024. The event is hosted by the Free Software Foundation (FSF), and brings together software developers, law and policy experts, activists, students, and computer users to learn skills, celebrate free software accomplishments, and face upcoming challenges. Newcomers are always welcome, and LibrePlanet 2024 will feature programming for all ages and experience levels.

*Please register in advance at <https://libreplanet.org/2024/>.*

wxie

17 April, 2024 11:30PM by Wensheng XIE

April 13, 2024

Simon Josefsson

Reproducible and minimal source-only tarballs

With the release of Libntlm version 1.8 the release tarball can be reproduced on several distributions. We also publish a signed minimal source-only tarball, produced by git-archive which is the same format used by Savannah, Codeberg, GitLab, GitHub and others. Reproducibility of both tarballs are tested continuously for regressions on GitLab through a CI/CD pipeline. If that wasn’t enough to excite you, the Debian packages of Libntlm are now built from the reproducible minimal source-only tarball. The resulting binaries are reproducible on several architectures.

What does that even mean? Why should you care? How you can do the same for your project? What are the open issues? Read on, dear reader…

This article describes my practical experiments with reproducible release artifacts, following up on my earlier thoughts that lead to discussion on Fosstodon and a patch by Janneke Nieuwenhuizen to make Guix tarballs reproducible that inspired me to some practical work.

Let’s look at how a maintainer release some software, and how a user can reproduce the released artifacts from the source code. Libntlm provides a shared library written in C and uses GNU Make, GNU Autoconf, GNU Automake, GNU Libtool and gnulib for build management, but these ideas should apply to most project and build system. The following illustrate the steps a maintainer would take to prepare a release:

git clone https://gitlab.com/gsasl/libntlm.git
cd libntlm
git checkout v1.8
./bootstrap
./configure
make distcheck
gpg -b libntlm-1.8.tar.gz

The generated files libntlm-1.8.tar.gz and libntlm-1.8.tar.gz.sig are published, and users download and use them. This is how the GNU project have been doing releases since the late 1980’s. That is a testament to how successful this pattern has been! These tarballs contain source code and some generated files, typically shell scripts generated by autoconf, makefile templates generated by automake, documentation in formats like Info, HTML, or PDF. Rarely do they contain binary object code, but historically that happened.

The XZUtils incident illustrate that tarballs with files that are not included in the git archive offer an opportunity to disguise malicious backdoors. I blogged earlier how to mitigate this risk by using signed minimal source-only tarballs.

The risk of hiding malware is not the only motivation to publish signed minimal source-only tarballs. With pre-generated content in tarballs, there is a risk that GNU/Linux distributions such as Trisquel, Guix, Debian/Ubuntu or Fedora ship generated files coming from the tarball into the binary *.deb or *.rpm package file. Typically the person packaging the upstream project never realized that some installed artifacts was not re-built through a typical autoconf -fi && ./configure && make install sequence, and never wrote the code to rebuild everything. This can also happen if the build rules are written but are buggy, shipping the old artifact. When a security problem is found, this can lead to time-consuming situations, as it may be that patching the relevant source code and rebuilding the package is not sufficient: the vulnerable generated object from the tarball would be shipped into the binary package instead of a rebuilt artifact. For architecture-specific binaries this rarely happens, since object code is usually not included in tarballs — although for 10+ years I shipped the binary Java JAR file in the GNU Libidn release tarball, until I stopped shipping it. For interpreted languages and especially for generated content such as HTML, PDF, shell scripts this happens more than you would like.

Publishing minimal source-only tarballs enable easier auditing of a project’s code, to avoid the need to read through all generated files looking for malicious content. I have taken care to generate the source-only minimal tarball using git-archive. This is the same format that GitLab, GitHub etc offer for the automated download links on git tags. The minimal source-only tarballs can thus serve as a way to audit GitLab and GitHub download material! Consider if/when hosting sites like GitLab or GitHub has a security incident that cause generated tarballs to include a backdoor that is not present in the git repository. If people rely on the tag download artifact without verifying the maintainer PGP signature using GnuPG, this can lead to similar backdoor scenarios that we had for XZUtils but originated with the hosting provider instead of the release manager. This is even more concerning, since this attack can be mounted for some selected IP address that you want to target and not on everyone, thereby making it harder to discover.

With all that discussion and rationale out of the way, let’s return to the release process. I have added another step here:

make srcdist
gpg -b libntlm-1.8-src.tar.gz

Now the release is ready. I publish these four files in the Libntlm’s Savannah Download area, but they can be uploaded to a GitLab/GitHub release area as well. These are the SHA256 checksums I got after building the tarballs on my Trisquel 11 aramo laptop:

91de864224913b9493c7a6cec2890e6eded3610d34c3d983132823de348ec2ca  libntlm-1.8-src.tar.gz
ce6569a47a21173ba69c990965f73eb82d9a093eb871f935ab64ee13df47fda1  libntlm-1.8.tar.gz

So how can you reproduce my artifacts? Here is how to reproduce them in a Ubuntu 22.04 container:

podman run -it --rm ubuntu:22.04
apt-get update
apt-get install -y --no-install-recommends autoconf automake libtool make git ca-certificates
git clone https://gitlab.com/gsasl/libntlm.git
cd libntlm
git checkout v1.8
./bootstrap
./configure
make dist srcdist
sha256sum libntlm-*.tar.gz

You should see the exact same SHA256 checksum values. Hooray!

This works because Trisquel 11 and Ubuntu 22.04 uses the same version of git, autoconf, automake, and libtool. These tools do not guarantee the same output content for all versions, similar to how GNU GCC does not generate the same binary output for all versions. So there is still some delicate version pairing needed.

Ideally, the artifacts should be possible to reproduce from the release artifacts themselves, and not only directly from git. It is possible to reproduce the full tarball in a AlmaLinux 8 container – replace almalinux:8 with rockylinux:8 if you prefer RockyLinux:

podman run -it --rm almalinux:8
dnf update -y
dnf install -y make wget gcc
wget https://download.savannah.nongnu.org/releases/libntlm/libntlm-1.8.tar.gz
tar xfa libntlm-1.8.tar.gz
cd libntlm-1.8
./configure
make dist
sha256sum libntlm-1.8.tar.gz

The source-only minimal tarball can be regenerated on Debian 11:

podman run -it --rm debian:11
apt-get update
apt-get install -y --no-install-recommends make git ca-certificates
git clone https://gitlab.com/gsasl/libntlm.git
cd libntlm
git checkout v1.8
make -f cfg.mk srcdist
sha256sum libntlm-1.8-src.tar.gz 

As the Magnus Opus or chef-d’œuvre, let’s recreate the full tarball directly from the minimal source-only tarball on Trisquel 11 – replace docker.io/kpengboy/trisquel:11.0 with ubuntu:22.04 if you prefer.

podman run -it --rm docker.io/kpengboy/trisquel:11.0
apt-get update
apt-get install -y --no-install-recommends autoconf automake libtool make wget git ca-certificates
wget https://download.savannah.nongnu.org/releases/libntlm/libntlm-1.8-src.tar.gz
tar xfa libntlm-1.8-src.tar.gz
cd libntlm-v1.8
./bootstrap
./configure
make dist
sha256sum libntlm-1.8.tar.gz

Yay! You should now have great confidence in that the release artifacts correspond to what’s in version control and also to what the maintainer intended to release. Your remaining job is to audit the source code for vulnerabilities, including the source code of the dependencies used in the build. You no longer have to worry about auditing the release artifacts.

I find it somewhat amusing that the build infrastructure for Libntlm is now in a significantly better place than the code itself. Libntlm is written in old C style with plenty of string manipulation and uses broken cryptographic algorithms such as MD4 and single-DES. Remember folks: solving supply chain security issues has no bearing on what kind of code you eventually run. A clean gun can still shoot you in the foot.

Side note on naming: GitLab exports tarballs with pathnames libntlm-v1.8/ (i.e.., PROJECT-TAG/) and I’ve adopted the same pathnames, which means my libntlm-1.8-src.tar.gz tarballs are bit-by-bit identical to GitLab’s exports and you can verify this with tools like diffoscope. GitLab name the tarball libntlm-v1.8.tar.gz (i.e., PROJECT-TAG.ARCHIVE) which I find too similar to the libntlm-1.8.tar.gz that we also publish. GitHub uses the same git archive style, but unfortunately they have logic that removes the ‘v’ in the pathname so you will get a tarball with pathname libntlm-1.8/ instead of libntlm-v1.8/ that GitLab and I use. The content of the tarball is bit-by-bit identical, but the pathname and archive differs. Codeberg (running Forgejo) uses another approach: the tarball is called libntlm-v1.8.tar.gz (after the tag) just like GitLab, but the pathname inside the archive is libntlm/, otherwise the produced archive is bit-by-bit identical including timestamps. Savannah’s CGIT interface uses archive name libntlm-1.8.tar.gz with pathname libntlm-1.8/, but otherwise file content is identical. Savannah’s GitWeb interface provides snapshot links that are named after the git commit (e.g., libntlm-a812c2ca.tar.gz with libntlm-a812c2ca/) and I cannot find any tag-based download links at all. Overall, we are so close to get SHA256 checksum to match, but fail on pathname within the archive. I’ve chosen to be compatible with GitLab regarding the content of tarballs but not on archive naming. From a simplicity point of view, it would be nice if everyone used PROJECT-TAG.ARCHIVE for the archive filename and PROJECT-TAG/ for the pathname within the archive. This aspect will probably need more discussion.

Side note on git archive output: It seems different versions of git archive produce different results for the same repository. The version of git in Debian 11, Trisquel 11 and Ubuntu 22.04 behave the same. The version of git in Debian 12, AlmaLinux/RockyLinux 8/9, Alpine, ArchLinux, macOS homebrew, and upcoming Ubuntu 24.04 behave in another way. Hopefully this will not change that often, but this would invalidate reproducibility of these tarballs in the future, forcing you to use an old git release to reproduce the source-only tarball. Alas, GitLab and most other sites appears to be using modern git so the download tarballs from them would not match my tarballs – even though the content would.

Side note on ChangeLog: ChangeLog files were traditionally manually curated files with version history for a package. In recent years, several projects moved to dynamically generate them from git history (using tools like git2cl or gitlog-to-changelog). This has consequences for reproducibility of tarballs: you need to have the entire git history available! The gitlog-to-changelog tool also output different outputs depending on the time zone of the person using it, which arguable is a simple bug that can be fixed. However this entire approach is incompatible with rebuilding the full tarball from the minimal source-only tarball. It seems Libntlm’s ChangeLog file died on the surgery table here.

So how would a distribution build these minimal source-only tarballs? I happen to help on the libntlm package in Debian. It has historically used the generated tarballs as the source code to build from. This means that code coming from gnulib is vendored in the tarball. When a security problem is discovered in gnulib code, the security team needs to patch all packages that include that vendored code and rebuild them, instead of merely patching the gnulib package and rebuild all packages that rely on that particular code. To change this, the Debian libntlm package needs to Build-Depends on Debian’s gnulib package. But there was one problem: similar to most projects that use gnulib, Libntlm depend on a particular git commit of gnulib, and Debian only ship one commit. There is no coordination about which commit to use. I have adopted gnulib in Debian, and add a git bundle to the *_all.deb binary package so that projects that rely on gnulib can pick whatever commit they need. This allow an no-network GNULIB_URL and GNULIB_REVISION approach when running Libntlm’s ./bootstrap with the Debian gnulib package installed. Otherwise libntlm would pick up whatever latest version of gnulib that Debian happened to have in the gnulib package, which is not what the Libntlm maintainer intended to be used, and can lead to all sorts of version mismatches (and consequently security problems) over time. Libntlm in Debian is developed and tested on Salsa and there is continuous integration testing of it as well, thanks to the Salsa CI team.

Side note on git bundles: unfortunately there appears to be no reproducible way to export a git repository into one or more files. So one unfortunate consequence of all this work is that the gnulib *.orig.tar.gz tarball in Debian is not reproducible any more. I have tried to get Git bundles to be reproducible but I never got it to work — see my notes in gnulib’s debian/README.source on this aspect. Of course, source tarball reproducibility has nothing to do with binary reproducibility of gnulib in Debian itself, fortunately.

One open question is how to deal with the increased build dependencies that is triggered by this approach. Some people are surprised by this but I don’t see how to get around it: if you depend on source code for tools in another package to build your package, it is a bad idea to hide that dependency. We’ve done it for a long time through vendored code in non-minimal tarballs. Libntlm isn’t the most critical project from a bootstrapping perspective, so adding git and gnulib as Build-Depends to it will probably be fine. However, consider if this pattern was used for other packages that uses gnulib such as coreutils, gzip, tar, bison etc (all are using gnulib) then they would all Build-Depends on git and gnulib. Cross-building those packages for a new architecture will therefor require git on that architecture first, which gets circular quick. The dependency on gnulib is real so I don’t see that going away, and gnulib is a Architecture:all package. However, the dependency on git is merely a consequence of how the Debian gnulib package chose to make all gnulib git commits available to projects: through a git bundle. There are other ways to do this that doesn’t require the git tool to extract the necessary files, but none that I found practical — ideas welcome!

Finally some brief notes on how this was implemented. Enabling bootstrappable source-only minimal tarballs via gnulib’s ./bootstrap is achieved by using the GNULIB_REVISION mechanism, locking down the gnulib commit used. I have always disliked git submodules because they add extra steps and has complicated interaction with CI/CD. The reason why I gave up git submodules now is because the particular commit to use is not recorded in the git archive output when git submodules is used. So the particular gnulib commit has to be mentioned explicitly in some source code that goes into the git archive tarball. Colin Watson added the GNULIB_REVISION approach to ./bootstrap back in 2018, and now it no longer made sense to continue to use a gnulib git submodule. One alternative is to use ./bootstrap with --gnulib-srcdir or --gnulib-refdir if there is some practical problem with the GNULIB_URL towards a git bundle the GNULIB_REVISION in bootstrap.conf.

The srcdist make rule is simple:

git archive --prefix=libntlm-v1.8/ -o libntlm-1.8-src.tar.gz HEAD

Making the make dist generated tarball reproducible can be more complicated, however for Libntlm it was sufficient to make sure the modification times of all files were set deterministically to the timestamp of the last commit in the git repository. Interestingly there seems to be a couple of different ways to accomplish this, Guix doesn’t support minimal source-only tarballs but rely on a .tarball-timestamp file inside the tarball. Paul Eggert explained what TZDB is using some time ago. The approach I’m using now is fairly similar to the one I suggested over a year ago. If there are problems because all files in the tarball now use the same modification time, there is a solution by Bruno Haible that could be implemented.

Side note on git tags: Some people may wonder why not verify a signed git tag instead of verifying a signed tarball of the git archive. Currently most git repositories uses SHA-1 for git commit identities, but SHA-1 is not a secure hash function. While current SHA-1 attacks can be detected and mitigated, there are fundamental doubts that a git SHA-1 commit identity uniquely refers to the same content that was intended. Verifying a git tag will never offer the same assurance, since a git tag can be moved or re-signed at any time. Verifying a git commit is better but then we need to trust SHA-1. Migrating git to SHA-256 would resolve this aspect, but most hosting sites such as GitLab and GitHub does not support this yet. There are other advantages to using signed tarballs instead of signed git commits or git tags as well, e.g., tar.gz can be a deterministically reproducible persistent stable offline storage format but .git sub-directory trees or git bundles do not offer this property.

Doing continous testing of all this is critical to make sure things don’t regress. Libntlm’s pipeline definition now produce the generated libntlm-*.tar.gz tarballs and a checksum as a build artifact. Then I added the 000-reproducability job which compares the checksums and fails on mismatches. You can read its delicate output in the job for the v1.8 release. Right now we insists that builds on Trisquel 11 match Ubuntu 22.04, that PureOS 10 builds match Debian 11 builds, that AlmaLinux 8 builds match RockyLinux 8 builds, and AlmaLinux 9 builds match RockyLinux 9 builds. As you can see in pipeline job output, not all platforms lead to the same tarballs, but hopefully this state can be improved over time. There is also partial reproducibility, where the full tarball is reproducible across two distributions but not the minimal tarball, or vice versa.

If this way of working plays out well, I hope to implement it in other projects too.

What do you think? Happy Hacking!

13 April, 2024 04:44PM by simon

April 07, 2024

stow @ Savannah

GNU Stow 2.4.0 released

Stow 2.4.0 has been released. This release contains some much-wanted bug-fixes — specifically, fixing the --dotfiles option to work with dot-foo directories, and avoiding a spurious warning when unstowing. There were also very many clean-ups and improvements, mostly internal and not visible to users. See http://git.savannah.gnu.org/cgit/stow.git/tree/NEWS for more details.

07 April, 2024 11:22PM by Adam Spiers

April 01, 2024

Simon Josefsson

Towards reproducible minimal source code tarballs? On *-src.tar.gz

While the work to analyze the xz backdoor is in progress, several ideas have been suggested to improve the software supply chain ecosystem. Some of those ideas are good, some of the ideas are at best irrelevant and harmless, and some suggestions are plain bad. I’d like to attempt to formalize two ideas, which have been discussed before, but the context in which they can be appreciated have not been as clear as it is today.

  1. Reproducible tarballs. The idea is that published source tarballs should be possible to reproduce independently somehow, and that this should be continuously tested and verified — preferrably as part of the upstream project continuous integration system (e.g., GitHub action or GitLab pipeline). While nominally this looks easy to achieve, there are some complex matters in this, for example: what timestamps to use for files in the tarball? I’ve brought up this aspect before.
  2. Minimal source tarballs without generated vendor files. Most GNU Autoconf/Automake-based tarballs pre-generated files which are important for bootstrapping on exotic systems that does not have the required dependencies. For the bootstrapping story to succeed, this approach is important to support. However it has become clear that this practice raise significant costs and risks. Most modern GNU/Linux distributions have all the required dependencies and actually prefers to re-build everything from source code. These pre-generated extra files introduce uncertainty to that process.

My strawman proposal to improve things is to define new tarball format *-src.tar.gz with at least the following properties:

  1. The tarball should allow users to build the project, which is the entire purpose of all this. This means that at least all source code for the project has to be included.
  2. The tarballs should be signed, for example with PGP or minisign.
  3. The tarball should be possible to reproduce bit-by-bit by a third party using upstream’s version controlled sources and a pointer to which revision was used (e.g., git tag or git commit).
  4. The tarball should not require an Internet connection to download things.
    • Corollary: every external dependency either has to be explicitly documented as such (e.g., gcc and GnuTLS), or included in the tarball.
    • Observation: This means including all *.po gettext translations which are normally downloaded when building from version controlled sources.
  5. The tarball should contain everything required to build the project from source using as much externally released versioned tooling as possible. This is the “minimal” property lacking today.
    • Corollary: This means including a vendored copy of OpenSSL or libz is not acceptable: link to them as external projects.
    • Open question: How about non-released external tooling such as gnulib or autoconf archive macros? This is a bit more delicate: most distributions either just package one current version of gnulib or autoconf archive, not previous versions. While this could change, and distributions could package the gnulib git repository (up to some current version) and the autoconf archive git repository — and packages were set up to extract the version they need (gnulib’s ./bootstrap already supports this via the –gnulib-refdir parameter), this is not normally in place.
    • Suggested Corollary: The tarball should contain content from git submodule’s such as gnulib and the necessary Autoconf archive M4 macros required by the project.
  6. Similar to how the GNU project specify the ./configure interface we need a documented interface for how to bootstrap the project. I suggest to use the already well established idiom of running ./bootstrap to set up the package to later be able to be built via ./configure. Of course, some projects are not using the autotool ./configure interface and will not follow this aspect either, but like most build systems that compete with autotools have instructions on how to build the project, they should document similar interfaces for bootstrapping the source tarball to allow building.

If tarballs that achieve the above goals were available from popular upstream projects, distributions could more easily use them instead of current tarballs that include pre-generated content. The advantage would be that the build process is not tainted by “unnecessary” files. We need to develop tools for maintainers to create these tarballs, similar to make dist that generate today’s foo-1.2.3.tar.gz files.

I think one common argument against this approach will be: Why bother with all that, and just use git-archive outputs? Or avoid the entire tarball approach and move directly towards version controlled check outs and referring to upstream releases as git URL and commit tag or id. One problem with this is that SHA-1 is broken, so placing trust in a SHA-1 identifier is simply not secure. Another counter-argument is that this optimize for packagers’ benefits at the cost of upstream maintainers: most upstream maintainers do not want to store gettext *.po translations in their source code repository. A compromise between the needs of maintainers and packagers is useful, so this *-src.tar.gz tarball approach is the indirection we need to solve that. Update: In my experiment with source-only tarballs for Libntlm I actually did use git-archive output.

What do you think?

01 April, 2024 10:28AM by simon

March 31, 2024

parallel @ Savannah

GNU Parallel 20240322 ('Sweden') released [stable]

GNU Parallel 20240322 ('Sweden') has been released. It is available for download at: lbry://@GnuParallel:4

Quote of the month:

   GNU parallel ftw
    -- hostux.social/@rmpr @_paulmairo@twitter

New in this release:

  • Bug fixes and man page updates.


GNU Parallel - For people who live life in the parallel lane.

If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.


About GNU Parallel


GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

For example you can run this to convert all jpeg files into png and gif files and have a progress bar:

  parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif

Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:

  find . -name '*.jpg' |
    parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

    $ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
       fetch -o - http://pi.dk/3 ) > install.sh
    $ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
    12345678 883c667e 01eed62f 975ad28b 6d50e22a
    $ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
    cc21b4c9 43fd03e9 3ae1ae49 e28573c0
    $ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
    79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
    fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
    $ bash install.sh

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

  • Give a demo at your local user group/team/colleagues
  • Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
  • Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
  • Request or write a review for your favourite blog or magazine
  • Request or build a package for your favourite distribution (if it is not already there)
  • Invite me for your next conference


If you use programs that use GNU Parallel for research:

  • Please cite GNU Parallel in you publications (use --citation)


If GNU Parallel saves you money:



About GNU SQL


GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.


About GNU Niceload


GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

31 March, 2024 09:11PM by Ole Tange

March 30, 2024

poke @ Savannah

poke-elf 1.0 released

I am happy to announce the first release of poke-elf, version 1.0.

The tarball poke-elf-1.0.tar.gz is now available at
https://ftp.gnu.org/gnu/poke/poke-elf-1.0.tar.gz.

> poke-elf (https://jemarch.net/poke-elf) is a full-fledged GNU poke pickle for editing ELF object
> files, executables, shared libraries and core dumps.  It supports
> many architectures and extensions.
>
> This pickle is part of the GNU poke project.
>
> GNU poke (https://jemarch.net/poke) is an interactive, extensible
> editor for binary data.  Not limited to editing basic entities such
> as bits and bytes, it provides a full-fledged procedural,
> interactive programming language designed to describe data
> structures and to operate on them.


Please send us comments, suggestions, bug reports, patches,
questions, complaints, bitcoins, or whatever, to poke-devel@gnu.org.

Happy ELF poking!

---

Jose E. Marchesi
Frankfurt am Main
30 March 2024


30 March, 2024 07:08PM by Jose E. Marchesi