Jdoe’s Throwaway

I’ve been recently working, as part of my thesis, with the AQuoSA architecture.

Since I’m a gentoo user and no ebuild did exist (at least, I did not find them), I’ve created an ebuild repository.

To use it you need bzr installed, and yes, it’s *really* a good thing if you read the AQuoSA doc before installing any of these ebuilds.

$ bzr co http://bzr.cnglab.net/aquosa-repo

as of now included ebuilds are:

sys-kernel/aquosa-sources : full linux sources with generic scheduler patch
sys-apps/aquosa-qosres    : qres program and aquosa headers, plus the qresmod linux module and aquosa init script
sys-apps/aquosa-qosmgr    : qmgr program
www-apache/mod_reserve    : apache2 module to provide QoS thought the AQuoSA infrastructure
media-sound/jack-audio-connection-kit : as of now it's unrelated to AQuoSA, but it's a WIP ebuild for jack2 svn
                                        with support for the pipelining branch. This ebuild is based on the proaudio
                                        repository ebuild.

Please report back to me every problem you will (yes, you will ) encounter using these ebuilds.

Instructions for paludis:

create a aquosa.conf file in the /etc/paludis/repositories/ directory with the following content

location = ${ROOT}/var/paludis/repositories/aquosa
sync = bzr+http://bzr.cnglab.net/aquosa-repo
master_repository = gentoo
format = ebuild
write_cache = /var/cache/paludis/metadata
names_cache = /var/cache/paludis/names

obviously change the location path to match your system settings.

then do the first-time sync of the repository:

$ paludis --sync x-aquosa

Instructions for portage:

sorry, I don’t use portage since I switched to paludis, If someone wants to contribute install instructions for portage he/she is welcome.

Ok, in the previous post I’ve shown a way to run web2py behind apache  using mod_wsgi.

The main “problem” still to be addressed is: URIs. Assume you have 3 website which implies (in the common case) 3 web2py application,

1. www.primo.it with the we2bpy app called primo
2. www.secondo.it , appname is secondo
3. www.terzo.it, appname is terzo.

(ok, maybe I lack of fantasy … )

Now we ended up having a vhost with

ServerName web2py.localhost

so all of our 3 applications are at

web2py.localhost/primo/default/index, web2py.localhost/secondo/default/index, web2py.localhost/terzo/default/index. It should work, but it’s not what we want.

The simplest thing here is to add

ServerAlias www.primo.it www.secondo.it www.terzo.it

to the vhost conf file. This works, but has a couple of disadvantages:

1. application name is still in the url (i.e. www.primo.it/primo/default/index)
2. secondo and terzo are available from primo’s url and vice-versa (www.primo.it/secondo/default/index shows secondo application)

Here enters mod_rewrite. Leave the ServerAlias in place, as they are needed.

Disclaimer I’m **NOT** a mod_rewrite guru -as you’ll see-, and I really think that there are better ways to accomplish this task: suggestions are welcome ^^

Starting with www.primo.it, add this at the beginning of the vhost file, just after the DocumentRoot directive:

RewriteEngine On

RewriteCond %{HTTP_HOST} =www.primo.it [NC, OR]
RewriteCond %{HTTP_HOST} =primo.it [NC]
RewriteRule /primo/(.*) /$1 [N]

RewriteCond %{HTTP_HOST} =www.primo.it [NC, OR]
RewriteCond %{HTTP_HOST} =primo.it [NC]
RewriteRule ^/(.*) /primo/$1 [PT,L]

The first “block” is needed to fix “links” in web2py applications, so that if you are using URL() function it continues to work. The problem it addresses is that we don’t want application name in URLs buth URL() keeps adding it. So we remove it;  the [N] flag after  the first RewriteRule means “restart from the beginning after applying” (the downside is that this is an external redirect )

The second “block” is needed to do the actual work: it transparently add the app name to the request, but the url in user browser remains without. So one user accessing www.primo.it/default/index arrives to wsgi as if he/she was calling www.primo.it/primo/default/index (and the user still have www.primo.it/default/index) in his/her location bar. The [L] flags means “last” (i.e. stop processing rewrites) and [PT] means “jump directly to the next alias handler”, which is the WSGIScriptAlias defined in previous posts.

Repeat this two blocks (RewriteEngine On is needed only the first time) for every application you have (you can do includes in separate files if you wish)  and you end up having

• www.primo.it that redirect to www.primo.it/default/index.
• www.primo.it/secondo/default/index still working
• web2py.localhost/primo/default/index still working
• https (i.e. management) only on web2py.localhost. If you want https support for primo.it you should add the rewrite rules event to the https section, (or -better- include two times the same file)

Hope this helps… and if you have better options or ideas write a comment

Last thing about the RewriteCond: yes, I already know that it can be done with a regex instead of using the =www.primo.it AND rewriting the condition without www. but I like this way far more as it’s simpler and much more understandable.

Web2py

Install web2py

First, what you need is to install web2py. This could be as simple as download/unzip (in this case do it yourself), but I think a better option is to make revision control system do the “update job” for us

Since I’m a bazaar user and web2py has a branch on launchpad let’s use it.  The branch is a development branch, so check the log to see a revision which is marked as release. At the time of writing revision 555 is marked as release 1.56.4 in the logs:

555.  By Massimo Di Pierro <email address hidden> on 2009-02-23
                1.56.4

So cd in the directory where you wish to store you web2py applications and get the code (remember the revision we checked above)

$ cd /var/www/
$ bzr branch -r 555 lp:~mdipierro/web2py/devel web2py

Next time you want to upgrade web2py to the next release simply check the new revision and pull your changes.

$ cd /var/www/web2py/
$ bzr pull -r XYZ


Configure web2py

You need to chown the directory so that the webserver user can write *.pyc files.
You can override the user in the configuration, so you basically have to choices:

• The user running apache (apache on gentoo, www-data on ubuntu)
• Use your user (useful only for development, not suitable in production)

Since here we are talking about production, let’s take the first option

$ groupadd web2py
$ chown -R apache:web2py /var/www/web2py
$ chmod -R g+w /var/www/web2py

The web2py group is optional, of course.
Now start web2py “normally” so that it create required files. Insert a password for the admin page, and remember it. Stop web2py by hitting CTRL+C

$ python web2py
...
...
CTRL+C


Now, it should be present two files in the web2py directory:

• options_std.py
• parameters_8000.py

Edit options_std.py and ensure that the line

password = '<recycle>'

is present (it really should)
Create some symlinks:

ln -s options_std.py options.py
ln -s parameters_8000.py parameters_80.py
ln -s parameters_8000.py parameters_443.py

Ok, we’re done with web2py configuration, let’s concentrate on apache and mod_wsgi.

Apache

Installation

Apache install it’s not covered here. Apache and apache ssl is assumed to work before following these istructions

Install required modules

Install mod_wsgi and enable it
On Ubuntu:

$ sudo apt-get install libapache2-mod-wsgi
$ sudo a2enmod wsgi

On Gentoo:

sudo paludis -i www-apache/mod_wsgi
sudo vim /etc/conf.d/apache2
--> add -D WSGI to APACHE2_OPTS variable


Configure vhost

Web2py with wsgi require that you run it in it’s own vhost.
For this to work you have to use a different domain from “localhost”
A dirty hack is to add a new domain name for localhost in /etc/hosts and use it for web2py
open /etc/hosts and add an alias for 127.0.0.1. Obviously on production system you should not have this problem
and you can use a “real” domain name

127.0.0.1 ... ... web2py.localhost

“… …” is not really there in the file, it’s just a placeholder for things already written on that row.

Next, create a new vhost file:

Ubuntu

vim /etc/apache2/sites-available/web2py

Gentoo

vim /etc/apache2/vhost.d/web2py.conf

Vhost file

In the editor, write the following in this file
** CHECK ALL PATHS, SPECIALLY FOR CERTIFICATES **
sample_vhost.conf

OK, almost done. On ubuntu enable the vhost

sudo a2ensite web2py

next, (on ubuntu) add

NameVirtualHost *:443

after the line

<IfModule mod_ssl.c>

in file /etc/apache2/sites-enabled/default-ssl

then restart apache (do not reload, need restart)
sudo /etc/init.d/apache2 restart

go to http://web2py.localhost/ and you should see the welcome page.
https://web2py.localhost/ should work too.

Next article will introduce a bit of mod_rewrite work to manage default application on a per-domain base.

Tempo fa installai openssh5 sul mio server perchè necessitavo della nuova feature introdotta in quella versione, che permette in pochi passi e senza troppi problemi di avere un chroot selettivo per alcuni utenti, in modo da poter abilitare SFTP e disabilitare il login. Veniva (e viene) usato per bzr e devo dire che è molto comodo: si abilita in un attimo

Una volta installato openssh5 (io ho openssh-5.0_p1) basta aggiungere le seguenti righe al file /etc/ssh/sshd_config

Match group bzrsftp
    ForceCommand internal-sftp
    ChrootDirectory /srv/bzr/%u

e questo limita tutti gli utenti che fanno parte del gruppo bzrsftp a usare sftp (i tentativi di login per avere una shell falliscono) e esegue il chroot nella loro directory: in questo modo vedono solo i loro file (il repo) e tutti vissero felici e contenti :p

Oggi avevo bisogno invece di avere un utente che potesse loggarsi per fare un reverse tunnel (per esporre un servizio hostato nella rete fastweb… discorso lungo, cmq nulla di che ).
Solo che non volevo che questo utente (che si connette con la chiave ma senza passphrase) avesse accesso a tutti i file e a tutti i programmi sul server…

Per prima cosa vi conviene creare l’account e testare il login con chiavi (mettendo la chiave pubblica nel file ~/.ssh/authorized_keys) e quando tutto funziona pensare al chroot.

Dopo un po’ di smattamenti sono arrivato alla conclusione.. intanto serve la conf di sshd
(sempre in /etc/ssh/sshd_config)

Match User tunnel
    ChrootDirectory /home/%u
    GatewayPorts clientspecified
    AllowTcpForwarding yes

Quindi è necessario copiare un po’ di eseguibili dentro la sua home directory
E’ un po’ noioso (andrebbe scriptato, ma non credo che mi ricapiti), assumiamo che la home dell’utente sia /home/tunnel:


# cp -rp /bin /home/tunnel/
# cp -rp /lib /home/tunnel/
# mkdir -p /home/tunnel/usr/{bin,lib} /home/tunnel/{etc,proc,dev} /home/tunnel/home/tunnel
# cp /usr/bin/{id,dircolors,ssh} /home/tunnel/usr/bin
# cp /usr/lib/{libcrypto.so.*,libssl.so.*} /home/tunnel/usr/lib
# cp -rp /home/tunnel/.ssh /home/tunnel/home/tunnel/.ssh
# cp -rp /etc/{bash*,resolv.conf} /home/tunnel/etc
# mount -t proc none /home/tunnel/proc
# mount -o bind /dev /home/tunnel/dev


Forse non è minimale come avrei voluto, ma è sufficente a far loggare l’utente e dargli qualche comando base… in più è possibile per l’utente fare i tunnel, e grazie alla direttiva GatewayPorts clientspecified può bindare i reverse tunnel su indirizzi diversi da 127.0.0.1 (in particolare mi l’ip pubblico del server)
Ricordatevi che i due comandi mount vanno dati all’avvio della macchina, se riavviate, quindi o li mettete nello script di avvio “local” o mettete le relative entry in /etc/fstab

Edit 20081116

C’è da aggiungere una cosa: le sessioni ssh vanno in timeout se non ci sono dati scambiati tra il client e il server.. E ovviamente quindi il tunnel cade se non ci sono dati che lo attraversano.

La soluzione è aggiungere

ClientAliveInterval 60

Che manda un keepalive al minuto che tiene su il tunnel. 60 è indicativo, potete anche aumentarlo o diminuirlo. ClientAliveInterval non può essere aggiunto (anche se mi sarebbe piaciuto) nel “blocco” di configurazioni per il nostro utente (Match User tunnell) ma va aggiunto nelle conf globali.

Usando bzr come VCS preferito, ho sempre trovato scomodo il file distribuito insieme al pacchetto su gentoo per abilitare la bash-completion. Infatti lo script completa solo qualche comando, non tutti, mentre io spesso non mi ricordo il nome preciso del comando e sono sempre a dare bzr help commands :p

E sicome sono pigro, mi sono riscritto lo scriptino: non so se funziona per voi, per me si. Beh, più che riscritto me lo sono “esteso” con i comandi che mancavano.

Lo script lo potete trovare qua: http://jdoe.asidev.com/files/bzr-autocompletion-20081103

per usarlo, basta che lo mettiate nella directory

~/.bash_completion.d/

con nome

bzr

Ah, ovviamente dovete avere bash-completion abilitata!

Ultimamente c’ho giocherellato un po’.. in verità lo uso per testare il cluster xen (si, faccio gentoo -> kvm->ubuntu->xen->gentoo, qualche problema? )

kvm è fantastico, se usato con lvm anzichè le immagini qcow per i file è anche piuttosto veloce. Più o menoc come virtualbox, che magari è più semplice da far andare per i più (cliky-clicky-cliky-kab00m) però non supporta PAE, e io ci devo far girare dentro centos o ubuntuserver, che hanno PAE nel kernel xen.. quindi it’s a no-go :p KVM, invece per ora non fa una piega

L’howto è finalizzato a configurare kvm per andare con le schede di rete in bridged mode, ovvero è come se la macchina virtuale fosse sulla lan insieme al resto dei pc di casa e al router.

In più, una seconda rete, tutta virtuale, per connettere le macchine tra loro (yum )

Lo trovate qua

* this post is in english. if you want to code javascript you have to know english, so it’s a non-sense to write this in italian *

Well for those of you that are approaching javascript and miss something like php include/require, python import ecc ecc this is somthing that works in a similar way
The “normal” way for doing this in javascript is to add a <script> tag

<script src="/path/to/file/to/include.js" type="text/javascript"></script>

Well, I don’t like this, cause when I’m editing the js file I don’t explicit which are the file’s deps.
So I’ve wrote a simple function for doing explicit including, lurking the MochiKit source code and basically copying from there.

var included_js = {};
var namespaceID ="ExampleCom";

function include(url,id)
{
	var kXULNSURI = "http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul";

	/* prevent re-include */
	if (url in included_js)
		return;

    if (document.documentElement &&
        document.documentElement.namespaceURI == kXULNSURI) {
        /* XUL based browsers */
        var s = document.createElementNS(kXULNSURI, 'script');
        s.setAttribute("id", namespaceID + "_" + id);
        s.setAttribute("src", url);
        s.setAttribute("type", "application/x-javascript");
        baseElem.parentNode.appendChild(s);
    } else {
    	/* not XHTML standard   */
    	document.write('<script src="' + url + '" type="text/javascript"></script>');
    }
    included_js[url] = true;
}

This function really doesn’t do anything that you can do yourself, it just prints out the <script> tag for you (or, if browser is mozilla, use the DOM/XUL to load the file)
I want to improve this function to:

• Avoid loading the script two time – updated!
• Permit a behavior similar to require

The usage is really as you can read from the short code above:

include('/path/to/file/to/include.js', 'FILEID');

Where FILEID is a string that identify the javascript file

[edit] 25/01/07 aggiornato l’howto [/edit]

Primo post puramente tecnico del blog. Primo (non credo di una lunga serie però…) howto.

Spero interessi. Per chi non lo sapesse KVM è un modulo del kernel prossimo all’inclusione nel kernel (2.6.20)

che viene utilizzato per sfruttare le instruzioni delle nuove cpu intel e amd: istruzioni di supporto alla virtualizzazione sopratutto sul lato performance.

Per ulteriori e piu precise info: Wikipedia
per un howto generico in inglese e altre cose c’è il sito di kvm :KVM official site

KVM gentoo howto