Journal of a developer

April 8th, 2010

As I promised, I’m here writing of this new experience :-)

Background: I started to learn how to develop 5 years ago. Since then, I made different programs, little projects with my friends, but I felt they were just “for me”. I was doing nice applications, learning lots of things every day, but I wasn’t actually helping anyone. So I started looking around in search of a good project to give my contribute. My boyfriend (a developer too) gave me the idea: GNU myserver, an application I’ve never heard about before. I gave a look at the program, the site, the code organization, and liked it. Few days after I proposed myself for contributing. That’s almost the end of the story!

But I have to say, this is a very good experience to me. Honest, I don’t have much to say, except that the people are very kind: they don’t mind helping me making my first step with this patch (especially with autotools… we don’t like each other! :D ). Before, I was talking about women in computing, but I don’t think that here we’re discouraged. I met many men and they mainly are nice; IMHO the reason we’re few is elsewhere. But I haven’t got many experience so this is just an impression. A thing I know for sure is that myserver is very open and doesn’t discriminate.

Lisa

ps = as usual, I have to thank my boyfriend. He’s always encouraging me and helping me debugging my code!

Configure the server using Scheme

February 24th, 2010

It was on my wish list for too long and I have finally started to work on it, now MyServer can load its main configuration from a Scheme file, virtual hosts and MIME types are still handled using XML files but works are in progress.
My idea was to implement this feature as a plugin but it couldn’t be handled as a “normal” plugin, as they are loaded after the server configuration is already loaded. As first step I added the possibility to specify as an argument to the myserver executable which plugins must be loaded before read the server configuration:

$ myserver --plugins=guile_conf:path/to/guile_conf.so

At this point I could load the guile_conf plugin and replace the configuration file handler with the Guile version.

I haven’t spend much time to think about a proper way to specify the configuration syntax, this is what we have in XML:


...
<DEFINE name="server.static_threads" value="10" />
<DEFINE name="http.default_file" foo="bar">
<DEFINE value="default.html"/>
<DEFINE value="default.htm"/>
<DEFINE value="index.html"/>
<DEFINE value="index.htm"/>
</DEFINE>
....

and here how it is translated to Scheme:

(define configuration '(
...
((name . "http.default_file")
(foo . "bar")
("default.html" "default.htm" "index.html" "index.htm"))
((name . "server.static_threads") "10")
...
))

Does it remind you something? :-)

I will probably change its structure, maintain attributes like a cons cells list before the value (or the children nodes) looks a bit tricky. Any suggestion?

The next move will be the support for MIME types and virtual hosts configuration files. Stay tuned!

Use Gnulib to improve portability

October 19th, 2009

It was already too much time in my todo list but I finally migrated MyServer to use Gnulib. For those who don’t know, as stated on its homepage “Gnulib is a central location for common GNU code, intended to be shared among GNU packages“. Differently from other libraries, it is shared at source level, that means you literally copy the source files that you need in your project tree and use them as you wish.
The nice thing of this approach is that you will not have to worry about different versions as you choose exactly which one to use; the very nice thing is that it will work in the same way on different systems.
Still there is much code that needs a review and eventually be dropped in favor of a Gnulib module, in other cases there is need of new C++ classes to wrap the Gnulib C APIs.

Many thanks to the Gnulib authors.

The official Gnulib page:
http://www.gnu.org/software/gnulib/

FSF internship – working with GNU MyServer

August 31st, 2009

Hi, my name is Marek, I study informatics (I’ve just ended my second year), I’m GNU Volunteer Webmaster and this summer I was an intern at FSF working on GNU MyServer. You can be curious why I’m not listed here along other interns – being first FSF intern I was accepted before FSF’s new internship program was started and I’m not part of it. I’m a pretty bad writer so please excuse my style and errors=).

For those that don’t know it, GNU MyServer is a web server as Apache or lighttpd. During my internship I’ve developed GUI application that lets you edit GNU MyServer’s configuration. Application is written in Python using GTK+ toolkit. I have used Python a lot and done some GUI programming before so it wasn’t completely new to me, but it was my first time using GTK+ which several times refused to do what I wanted=). Fortunately it is nicely documented and I managed to overcome all difficulties. I also had problems with not properly documented GNU MyServer configuration files, but Giuseppe Scrivano (GNU MyServer maintainer and my mentor) always helped me out=). I think that’s enough said about technical details, if you want to know more my work is now merged with main tree, so everyone can get it by pulling GNU MyServer’s master git branch from GNU Savannah.

This internship was a great experience, mostly because I’ve never worked on a bigger project with few years of history and active development team, but also because I’m huge Free Software fan, so being an intern at foundation that’s behind GNU operating system and the GPL was pretty exciting. As an informatics student I can tell you that Free Software is perfect for learning. I can’t imagine using software that I can’t study. I learnt much from Free Software and I wanted to contribute back to Free Software community for some time, but I’ve never knew how to start. Well, first time is behind me and I hope that it will be easier now to enter other projects=). During this internship I’ve learnt a few new things (GTK+, GNU Autotools) but I think that the most important aspect was learning how to work in a team and how to contribute to bigger projects when you can’t easily learn how internals work.

My internship is now over but still there are things to be done, both for my GUI application and for GNU MyServer. I’m going to continue contributing to GNU Myserver and I hope my code will be useful for others.

For those who have reached this point I have a prize, we all love screen shots don’t we?;)

Good luck to other FSF interns and happy hacking!
Marek Aaron Sapota (maarons at gnu.org)

Plugins Manager and myserver-get Tool

March 3rd, 2009

Well, it’s time also for me to write something here:) the last months I worked on rewriting and refactoring the plugins management in MyServer, in order to simplify the inner management, the development and the outer (users-side) management of plugins.

To reach the first two goals, I removed all the different kind of plugins were defined before, in order to obtain a system simpler to manage. The plugins manager now perform versions checking and in a better way dependencies checking. If someone wants to write some plugin for myserver, and he find some trouble with the Plugins Manager or things there can be some improvement on it, I encourage him to write any idea on the mailing list :)

To simplify the management of plugins for the end-users, I wrote a simply tool in python that will help the users to manage plugins. i called it myserver-get because of its similarities with the well known apt-get tool for packages management in debian-based GNU/Linux distributions.
It’s written in python, it works only with HTTP and FTP protocols, and implements the following commands:
- install: install a plugin or a list of plugins with all the dependencies.
- update: update the local plugins lists
- search: search for a plugin containing a word or a sentence in the description (use -v to have a more verbose response)
- source: download the sources of a plugin or a list of plugins.
- source-auto: download, compile and install a plugin or a list of plugins. this command doesn’t check for dependencies.
- remove: remove a plugin or a list of plugins.

a plugins repository should have the following features:
- a directory src with the source packages and a directory pub with the binary packages
- a file list.xml. You can find an example file that shows the simply syntax in the folder misc/myserver-get on the svn.

the source and binary packages should be created with the scons tool and the SConstruct file you can find in the plugins directory on the svn.

Myserver-get is not just a command-line tool, it’s also a library, in order to be easily included in a future GUI configuration and managing tool for MyServer.

MyServer fork server

November 9th, 2008

Different reasons convinced me to write a “fork server” for MyServer. Having an external process to manage forks will lead to these advantages:

  • Possibility to run the process with a different uid/gid even in the case MyServer has already setuid/setgid itself.
  • No access to the MyServer file descriptor table.
  • Much faster in the case fork is not implemented using pages copy-on-write

The fork server is initialized immediately after MyServer, it waits for connection on a TCP port and every time a process needs to be
spawned, two connections are opened to the fork server, they will be the stdin/stdout streams to the new process. These same connections are used by MyServer and the fork server to communicate before the new process is executed too. MyServer can be configured to change uid/gid after it is initialized but in this way it will not possible to set a different uid/gid for new processes because it has already lost root privileges.

A first test on the php process executed as a persistent FastCGI server shows how the file descriptor table is much smaller:

before (without the fork server):

php-cgi 0u TCP localhost:45275 (LISTEN)
php-cgi 1w 596 /dev/null
php-cgi 2u 3 /dev/pts/1
php-cgi 7u 26487 socket
php-cgi 8u 26488 socket
php-cgi 9u 596 /dev/null
php-cgi 10u 14631793 MyServerHTTP.err
php-cgi 11u TCP *:http-alt (LISTEN)
php-cgi 12u 14632158 MyServerHTTP.log
php-cgi 13u 14631624 MyServerHTTPS.err
php-cgi 14u TCP *:4443 (LISTEN)
php-cgi 15u 14632105 MyServerFTP.log
php-cgi 16u 14632107 MyServerFTP.err
php-cgi 17u TCP *:iprop (LISTEN)
php-cgi 18u 26610 can't identify protocol
php-cgi 19r 14811172 /tmp/myserver_2_3.tmp (deleted)

after (using the fork server):

php-cgi 0u TCP localhost:60391 (LISTEN)
php-cgi 1u TCP localhost:43856 (LISTEN)
php-cgi 2u 3 /dev/pts/1
php-cgi 3u TCP localhost:37548 (LISTEN)

Not that bad!

GNU TLS preferred instead of OpenSSL

November 3rd, 2008

Today I finally decided to work out this element of our task list. We are happy that now MyServer can be distributed under the GPLv3 terms license without any exception. Previously we were using OpenSSL for the HTTPS protocol, it was giving us some licensing troubles as we needed to add an exception to the original GPLv3 license.

The task was quite straightforward, GNU TLS is offering a nice abstraction layer on the OpenSSL APIs, only few functionalities were not present but it was easy to remove these incompabitilities.

For those interested to test it, the updated source code is available on the subversion repository.

MyServer agile development

October 19th, 2008

After many discussions we finally decided to move our development model to something more dynamic, embracing some agile techniques like unit testing and test driven development.
The most important feature for this model is that code is frequently committed to the central subversion repository and project developers are allowed to commit patches without a previous review. In fact the review phase is postponed after the patch is committed and notified on a commits mailing list, where all developers can look at it and eventually show mistakes; in any case a commit must be done only after the source code is validated by the test suite. The main disadvantage is that the repository can contain not stable code but on the other hand, it is always updated and contains latest changes. Branches are not frequent but they are used in case a commit or a group of commits will change important features, e.g. switching to an event driven scheduler that changed the core classes of MyServer.

TDD is not used for normal classes development but it is required for bug fixing, where it is possible. In this way we will be sure the same bug will not appear again in the future. Old classes still are missing tests, not for all of all adding tests is a trivial task, in many cases there is need to refactor a lot of code before it is possible to write tests.
Some reports to help the development are automatically generated from the latest repository version source code and they are available in the Development section of the MyServer web site. A particular attention is given to the code complexity, methods must be as shorter as possible to make testing easier and especially to make the source code more readable.

We decided to have a release only when all the planned objectives are completed but until it is achieved some alpha releases can be done on fixed dates. After a release, new objectives are planned for a new cycle.