This is work in progress. Do not link to this page, it might change!
Hardware and preparation | OS installation | Console and web monitoring | Network & services | Multimedia | Security
[update 2009-05-26] I see strange behavior when installing different distributions: some time I have my installation drive mapped as
/dev/sda (Debian), sometimes it's mapped as
/dev/sde (Ubuntu). It seems that
GRUB maps the drives differently (in different OS's) when more that one SATA controller is used. My mobo has 2 SATA controllers: Intel ("built-in") and Silicon Image ("card"), as per description in the above link. It might happen that, if the installation drive (primary) has a name such as
/dev/sde, when I remove a drive (let's say
/dev/hdb), the installation drive gets a different name upon reboot and no more boot... Got a tip from a guy to fix this by mounting partitions using UUID or label (
tune2fs -l):
- to see drives by UUID, type
ls -l /dev/disk/by-uuid/- to see drives by name (for sata only) type
ls -l /dev/sd*.
I'll try that later and post how it's done. Some links:
Permission in NTFS mount point (Ubuntu forum).
Another approach is to use labels:
Mounting File Systems Automatically with /etc/fstab.
Strange is also the way different distros assign network names... There is no consistency between distros. I have 2 network cards (one built-in, and a second one PCI with 2 adapters). In some distros eth0 is the built-in one, in others eth0 is the PCI adapter. Strange...
Server part of this machine is very complex, as such everything should be done having security in mind - I try to do basic security upon install of each server application, and add more advanced security setup explained in
Security Area (
to be added!). Server application might be publicly available services, like: PXE, DHCP, DNS, LAMP (Apache + MySQL + PHP), Email (POP3 + SMTP), LDAP, FTP, ISP Hosting Panel, FAX, SIP/VoIP/PBX and maybe others. In addition, I may add services for my personal use or testing, for which I don't want to use another machine to be always on: file sharing, photo albums, backup, virtualization, web monitoring (both hardware and network, for itself and other hosts), wireless access point, torrents etc.
Configuration & tasks
Network setup
At this moment, I have 3 network interfaces:
- LAN 1 = Internet Provider 1 (DHCP from provider)
- LAN 2 = Internet Provider 2 (PPPoE, DHCP from provider)
- LAN 3 = Private Network, wired & wireless, class 192.168.0.0/24 (DHCP server)
Links:
Debian Reference.
Boot message (console) error:
Configuring network interfaces... Interface 'lo' is already enabled.
Solution: not known. It doesn't mess anything so far. I didn't find anything on Google and I don't know where is this coming from. Just noticed it's there.
Syslog error (/var/log/boot):
if-up.d/mountnfs[dsl-provider]: waiting for interface eth2 before doing NFS mounts (warning).
Solution: none yet, but nothings wrong except delay on boot and this annoying message upon restart. Bug report:
#481028 (Debian).
PPPoE (RDS Link) - internet backup link
# aptitude install pppoeconf
# pppoeconf
It looks like
/etc/ppp/pppoe.conf file is not created by default. Running
pppoe-setup gives the following error:
Oh, dear, I don't see the file '/etc/ppp/pppoe.conf' anywhere
That is easy to fix by putting the default
pppoe.conf file from the source (see
this link.) I got this tip from the link
PPPoE Configuration. However, I coudn't make it work. Stil researching.
Helpful links:
PPP Over Ethernet (PPPoE) for Debian Linux (with pictures),
Network configuration @Debian Reference,
Masquerade and PPPoE.
Errors:
Sometimes, when I start RDS connection (PPPoE) using command
pon, I get the following error:
/usr/sbin/pppd: In file /etc/ppp/peers/provider: unrecognized option '/dev/modem'
The message seems to be correct,
/dev/modem does not exist. However, removing the line
/dev/modem in
/etc/ppp/peers/provider will not fix the issue - rather it won't start and the following message appear:
chat[3514]: abort on (BUSY)
chat[3514]: abort on (VOICE)
chat[3514]: abort on (NO CARRIER)
chat[3514]: abort on (NO DIALTONE)
chat[3514]: abort on (NO DIAL TONE)
chat[3514]: send (ATZ^M)
chat[3514]: expect (OK)
chat[3514]: alarm
chat[3514]: Failed
pppd[3500]: Connect script failed
Weired enough, started googleing. A few links I found useful...
Internet link (backup) / load balancing
Seems to be 2 solutions:
(1)
Spanning Multiple DSLs,
Multirouting with Linux,
Using Multiple network device to connect to the internet.
() Bonding -
Bonding (Port Trunking),
NIC Bonding On Debian Lenny,
NIC Bonding/Teaming /
wiki (Debian Sarge),
Ethernet Bridge + netfilter Howto,
Remote access: SSH
http://iulica.blogspot.ro/2013/03/ssh-login-with-putty-without-password.html
http://iulica.blogspot.ro/2013/03/remote-access-to-linux-via-ssh-using.html
To do:
- cut brute force attacks using tools such as
Fail2Ban,
sshdfilter,
DenyHosts,
Pam abl,
BlockHosts,
Samhain,
loginfailure.pl etc.; found a nice
Fail2Ban tutorial where we find out that
DenyHosts only blocks ssh, while
Fail2Ban can be configured for any program that writes login attempts to a log file
- configure ssh; add security to ssh login (max. 5-10 failed logins, then disable for a period of time). Links:
forum thread @linuxquestions.org,
sshblack -- Automatically BLACKLIST SSH attackers,
Securing SSH Using Denyhosts.
- ssh chrooted:
Chrooted SSH/SFTP Tutorial (Debian Lenny).
Links:
Turbocharge PuTTY.
Disable IPV6 (not used + unnecessary logging)
IPV6 is not used often on the internet (my provider don't use it) and it's useless.
It might be nice to play with it a little, when I have some free time. Few links to read about this:
How to Disable IPV6 in Ubuntu,
How to disable ipv6 in Lenny to avoid 1.0.0.0 in name resolution for AAAA type queries,
Disable IPV6 module on default kernels,
IPv6 in Debian.
Gateway/firewall
As a gateway, this box have to provide internet access to LAN and WiFi stations. I use masquerade for this. One simple way to set things up is adding the following lines to /etc/rc.local:
echo Starting NAT script...
# Turn on IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# Masquerade out via eth1 (first internet provider, using DHCP to get IP)
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
# Masquerade out via ppp0 (second internet provider, using PPPoE to get IP)
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
To do: firewall
Server applications
PXE (
preboot e
xecution
environment)
A PXE install server allows your client computers to boot and install a Linux distribution over the network, without the need of burning Linux iso images onto a CD/DVD, boot floppy images, etc. This is handy if your client computers don't have CD or floppy drives, or if you want to set up multiple computers at the same time (e.g. in a large enterprise), or simply because you want to save the money for the CDs/DVDs.
To do.
Links:
Setting Up A PXE Install Server For Multiple Linux Distributions On Debian Lenny.
DHCP
# aptitude install dhcp3-server
After installation, an error comes up:
Starting DHCP server: dhcpd3check syslog for diagnostics. failed!
For everything to work, configure network interface(s) (
/etc/network/interfaces) and dhcp server (
/etc/dhcp3/dhcpd.conf). After that start the daemon:
# /etc/init.d/dhcp3-server start
For some reason,
eth1 won't take IP via DHCP. This line in /etc/rc.local fix this issue:
DNS: BIND9 (or MyDNS alternative)
Pretty simple - follow up a
nice tutorial to install
bind9 in a
chrooted environment.
Links:
Secure BIND Template.
Problems: After installation and configuration, I got very slow internet connection (for network systems), as well as some annoying errors:
(1) A flood of messages get into logs like this:
named[...]: too many timeouts resolving '...' (in '...?): reducing the advertised EDNS UDP packet size to 512 octets
Solution: use "category edns-disabled { null; };" in your logging statement in named.conf.
(2) After installation, a flood of messages get into logs like this:
named[...]: lame server resolving 'hawk989s.com' (in 'hawk989s.com'?): DNS_IP#53
Solution: use "category lame-servers { null; };" in your logging statement in named.conf.
(3) After installation, a lot of messages get into logs like this:
named[...]: unexpected RCODE (SERVFAIL) resolving 'trumptouch.com/MX/IN': DNS_IP#53Solution: not yet. I only found
this link related. This is how to do it:
DNS configuration question (OpenSuse),
#275091 (RedHat),
Too many timeouts resolving / disabling EDNS messages.
(4) I found in
/var/log/syslog bunch of lines like this:
named[...]: client 66.238.93.161#13584: query (cache) './NS/IN' denied
Solutions (to check):
A solution to Potential DNS DDoS: named query (cache) ‘./NS/IN’ denied,
Annoying DNS Recursive queries,
Potential DNS DDOS,
Blocking Recursive Root DNS Queries with iptables.
More readings:
DNS queries for "." (root servers),
DNS Test,
Avoiding being used as DDoS reflector,
Loads of Query denied... is it an attack or a misconfiguration? [message board - question],
Loads of Query denied... is it an attack or a misconfiguration? [message board - answer],
Upward Referrals Considered Harmful.
Utils: Searching through repositories, I found the following:
-
bind9-doc (Documentation for BIND)
-
bindgraph (DNS statistics RRDtool frontend for BIND9)
-
smbind (PHP-based tool for managing DNS zones for BIND)
To do:
- ddns (dynamic DNS updates) for hosts connected via DHCP - would be nice to add and make it work for Windows workstations (
some claims it won't)
- configuration for a secondary DNS (when a secondary server will be available)
Links:
Fixing Reverse DNS.
Interesting projects:
-
dHelp - builds an HTML index of all documentation that's registered on a Debian system
Found a note about Apache: whenever the logrotate does its job, Apache restarts; to avoid
problems of Apache not starting back, check
Apache documentation.
Alternatives:
-
Cherokee - a very fast, flexible and easy to configure Web Server, supporting the widespread technologies nowadays: FastCGI, SCGI, PHP, CGI, SSI, TLS and SSL encrypted connections, Virtual hosts, Authentication, on the fly encoding, Load Balancing, Apache compatible log files, Data Base Balancing, Reverse HTTP Proxy, Traffic Shaper, Video Streaming and much more; it has a friendly web interface (
cherokee-admin - ports 9090, 9091 on localhost) for a no-hassle configuration of the server. Links:
Installing Cherokee With PHP5 And MySQL Support On Debian Lenny.
-
lighttpd - very fast and light web server; links:
Install PHP 5.3.0/Lighttpd On Debian (Lenny) With Imap, MySQL, Sqlite3 And ImageMagick Support,
Integrating eAccelerator Into PHP5 And Lighttpd (Debian Lenny),
Installing Lighttpd With PHP5 And MySQL Support On Debian Lenny.
-
nginx - high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server; links:
nginx wiki,
Installing Nginx With PHP5 And MySQL Support On Debian Lenny.
- Python SimpleHTTPServer (
# python -m SimpleHTTPServer 80 at the command prompt and your directory is online!). Links:
Serve current directory tree at http://$HOSTNAME:8000/,
Ad-hoc Server mit Python (Germain).
2. MySQL Database
To install MySQL server, we will use the following command
# aptitude install mysql-server mytop
MyTop is an a
top-like application used to monitor for MySQL.
Transferring database to the new server:
Moving MySQL database from one server to another,
Transferring database using rsync,
MySQL Administrator (Official),
HeidiSQL.
Other link(s):
MySQLTuner – High-performance MySQL tuning script,
Set Up Database Replication In MySQL,
Set Up A Load-Balanced MySQL Cluster,
Planet MySQL,
MySQL Performance Blog,
High Availability MySQL (blog).
3. PHP
to do
Email: Postfix + virtual users
Installing email server with virtual mailbox domains:
not much to write here - just followed HowToForge tutorial (probably the original is here), keeping also an eye on this howto and this one. What we get:
- email setup: Virtual Users And Domains With Postfix, Courier, MySQL And SquirrelMail
- antivirus/anti-spam: amavisd-new, SpamAssassin, And ClamAV
- spam filters: Razor, Pyzor, DCC - make SpamAssassin aware of them
Everything went smooth. It's rokin'!!
HowTo:
- check root mail: mutt -f /var/mail/root (some annoying questions 'though at start, to create /root/Mail folder, or move read mails to /root/mbox - I wish I know how to skip these; the good part: it does nothing if you don't want)
- check root mail: alpine -f /var/mail/root (it automatically creates in current profile the folder ~/mail and generates the config file .addressbook and .pinerc)
Note: pine is not part of Debian database (lenny/main/non-free). There is, instead, alpine.
WARNING: this tutorial worked well for one month, then it crashed (see bellow)! I'm looking for other solution for this task.
[update 2009-07-27] Suddenly, about 1 month after setting up the email part, it no longer worked. The problem is described on
HowToForge forum and unfortunately I was not able to fix the issue. However, strange things used to happen in the server itself, using that specific tutorial. For example, some of the log files won't logrotate properly: the file
/var/log/sysconfig had 0 (zero) bytes, while the real logs were added to
/var/log/sysconfig.1. After 3 weeks of waiting for smarter guys on the forum (at the same time searching myself for a solution), I had no other option than reinstall the server. I gave up this set up
Links:
Debian Lenny Postfix Howto,
Simple PHP mail wrapper,
Using Exim4 to send Messages through GMail on Debian Lenny,
mail function (php),
Virtual Users And Domains With Postfix, Courier And MySQL (+ SMTP-AUTH, Quota, SpamAssassin, ClamAV),
CentOS + Postfix + MySQL + TLS + SASL + Maildrop + SQLgrey + Amavisd + SpamAssassin + ClamAV + Courier-IMAP + Courier-POP3d + SqWebMail + Horde IMP,
Installing Horde Groupware Webmail Edition,
Using Postfix for Secure SMTP Gateways,
Howto: ISP-style Email Server with Debian-Etch and Postfix 2.3 (Postfix + Dovecot/POP3/SMTP + virtual users/MySQL + Amavis + Postgrey + Squirrelmail + Vacation/GoldFish - very detailed!),
Drupal + Postfix Integration Under Ubuntu 8.04 (Hardy).
To do next:
- mailing list system (mailman).
- web access to spam filter. Links:
WebUserInterfaces.
- other webmail frontend: RoundCube,
Errors in log:
[1] A lot of errors at the very beginning, and just a few after some time, in /var/log/mail.log
Jun 1 10:52:07 [host] postfix/trivial-rewrite[11658]: fatal: proxy:mysql:/etc/postfix/mysql-virtual_domains.cf(0,lock|fold_fix): table lookup problem
Got an answer on
this post: add to mysql user postfix@127.0.0.1.
Another fresh post (2009-06-05) waiting for answers
here.
[X] iRedMail - a shell script that lets you quickly deploy a full-featured mail solution in less than 2 minutes. Since iRedMail 0.5, it supports Debian 5.0.1 (both i386 and x86-64). Its objective is to make a linux mail server with the installation and configuration simple and easy to use. iRedMail supports both OpenLDAP and MySQL as backends for storing virtual domains and users; links:
iRedMai website,
Installation on Debian.
To do.
[X] Spam abuse
I find lately many informations about spam abuse and email servers compromised. That means the, whenever a new email server is configured,
special precautions have to take into account. Security should be the top priority, in order to have a clean and reputation-free email server. Otherwise, lot of headache will come with removal from spam block lists.
Spam block list checkers which, more or less, check against several lists at once:
Spam links,
MX Toolbox,
MultiRBL,
OpenRBL,
SpamHouse,
SenderBase,
GoogleGroups,
Google (query).
Spam block lists:
SORBS,
SpamCop,
UCEPROTECT-NETWORK (commercial).
LDAP
To do: agenda/contacts database
Links:
LDAP + Samba PDC + PAM/NSS on Debian Lenny HOWTO,
eGroupware + LDAP on Debian lenny mini-HOWTO,
iRedMail: Mail Server With LDAP, Postfix, RoundCube/SquirrelMail, Dovecot, ClamAV, SpamAssassin, Amavisd, DKIM, SPF On Debian (Lenny) 5.0.1,
FTP
For FTP to work faster in MASQ config use these lines in console, or add them to /etc/rc.local if you want to be available upon reboot:
# Make FTP faster
modprobe ip_nat_ftp
modprobe ip_conntrack_ftp
To do.
Links:
Debian Manual HowTo,
FTP behind NAT with TLS howto,
Virtual Hosting With Proftpd And MySQL (Incl. Quota) On Debian Lenny,
Virtual Hosting With PureFTPd And MySQL (Incl. Quota And Bandwidth Management) On Debian Lenny.
ISP Hosting Panel
To do.
Links:
Comparison of web hosting control panels.
Commercial applications:
-
cPanel ($425 yearly) - 3 tier structure (administrators, resellers and end-user website owners), with support for Apache, PHP, mySQL, Postgres, Perl, Python, and BIND, email (POP3, IMAP, SMTP). Several add-ons exist for an additional fee, the most notable being
Fantastico - a bundle of scripts which automate the installation of (not update-able) web applications such as WordPress, SMF, phpBB, Drupal, Joomla!, TikiWiki CMS/Groupware, Moodle and over 50 others.
-
DirectAdmin ($300) - graphical web-based web hosting control panel designed to make administration of websites easier
-
Hosting Controller - a complete web hosting automation control panel which is designed for web hosts to experience infinite hosting possibilities in cluster environment & lower their operational costs
-
InterWorx - a dedicated server control panel for both the system/cluster administrator and website administrator, made of the following modules:
NodeWorx (system administrators),
SiteWorx (website administrators), resellers.
-
Kloxo/Lxadmin - allows the host administrators to run either lighttpd/Apache or djbdns/bind and also provides graphical interface to switch between these programs without losing any data. Additionally, Kloxo Enterprise can transparently move web/mail/dns from one server running Apache to another running lighttpd. Kloxo comes integrated with
Installapp which is a bundle of approximately 130 web applications that can be installed to the hosted websites. It also supports Installatron (a third party application installer similar to
Fantastico) as a plugin. Links: VPS
Hosting Kloxo Control Panel's InstallApp.
-
Parallels H-Sphere - a web hosting Automation Control Panel for shared web hosting services, written in Java, supporting around 30 Payment gateways and 6 E-Payment Providers
-
Parallels Business Automation / HSPcomplete - allows service providers to offer customers a range of services, from shared Web hosting for small and medium-sized businesses to cluster configurations for large enterprises.
-
Parallels Plesk ($1400) (+
Parallels Plesk Sitebuilder - Blog, Image Gallery, Guestbook, eShop, SitePal, Forum, Feedback, Registration, RSS Reader, Voting, Script, Area Map, File Download, SiteMap, External Page, and Flash Intro)
OpenSource:
-
Baifox - very light control panel to managed services of a hosting service, developed with PHP, some javascript code, all configuration saved in sqlite3 database, under Lighttpd server.
-
ClarkConnect - a Linux distribution which transforms any standard PC into a dedicated firewall and Internet server/gateway, and managing using
WebConfig interface. Features include: Stateful Firewall (iptables), Networking and Security, Intrusion Detection and Prevention System (SNORT), Virtual Private Networking (PPTP, IPSec, OpenVPN), Web Proxy, with Content Filtering and Antivirus (Squid, DansGuardian), E-mail Services (Webmail, Postfix, SMTP, POP3/s, IMAP/s), Groupware (Kolab), Database and Web Server (easy to deploy LAMP stack), File and Print Services (Samba and CUPS), Flexshares (unified multi-protocol storage which currently employs CIFS, HTTP/S, FTP/S, and SMTP), MultiWAN (Internet fault tolerant design), Builtin Reports for system statistics and services (MRTG and others)
-
DTC (Domain Technologie Control) - a control panel aiming at commercial hosting
-
eBox Platform - an open source distribution and development framework, based on the Ubuntu Linux computer operating system, intended to manage services in a computer network, merging the following: Apache - webserver, mod_perl - CGI engine, OpenLDAP - Shared users and groups, OpenSSL - Cryptography, Netfilter/iptables - Firewall, NAT, BIND - Domain name system server, Squid - Web proxy-cache, DansGuardian - Content filtering, Postfix - Mail server, XMPP - Instant Messaging, Ntpd - Clock and date synchronization, OpenVPN - Virtual Private Network, Samba - Shared storage, Primary Domain Controller for Windows clients, Common Unix Printing System (CUPS) - Shared printers, Advanced Packaging Tool (APT) - Software installation and upgrade, Asterisk - Voice over Internet Protocol services, Snort - Network Intrusion-prevention system, eGroupware - Calendar sharing + address book + webmail, Dovecot - IMAP and POP3 server.
-
ehcp (Easy Hosting Control Panel) - links:
Set Up Ubuntu Server With EHCP (LAMP, DNS, FTP, Mail),
How To Quickly Set Up A Web Server Environment With EHCP.
-
gnupanel - a hosting control panel for Debian. As administrator you can create public and private hosting plans, accept Paypal, Cuentadigital and Dineromail payments, send messages to users, create redirections, use the integrated support ticket system, control bandwidth, disk space and define policies for accounts suspension. The users can use the habitual functions to create mail and FTP accounts, databases, directories protection, etc. In addition they can make payments, place domains in parking and activate or deactivate in each subdomain PHP directives like safe_mode and register_globals. GNUPanel stores its configuration on a postgreSQL 8.1 database and provides three web interfaces with SSL access at user, reseller and administrator level. Features Autoinstallation for Joomla, phpBB WordPress and osCommerce.
-
ISPConfig - allows for the user to manage internet services, such as web servers, FTP servers, database servers, DNS servers. It also allows for the configuration of firewalls, anti-virus, users and shell users, email autoresponders, spam filters and quota
-
ispCP (Internet Service Provider Control Panel) - completely based on the original open source (dead)
VHCS, it's a project founded to build a Multi Server Control and Administration Panel usable by any ISP
-
SME Server / e-smith - a Linux distribution based on CentOS, offering an operating system for computers used as web, file, email and database servers. It employs a comprehensive UI for all management-related tasks and is extensible through templates.
-
SysCP (System Control Panel) - software for administration of webservers based on and written in PHP and MySQL, with a web-based front end for customers of internet service providers, enabling them to manage their email addresses, domains and databases.
-
Webmin - web-based system configuration tool for OpenSolaris, Linux and other Unix-like systems (even Windows) to configure many operating system internals, such as users, disk quotas, services, configuration files etc., as well as modify and control many open source apps, such as the Apache HTTP Server, PHP, MySQL etc. (port 10 000). It can be expanded by installing modules such as
Usermin (webmail and other user-level tasks) and
Virtualmin (domain hosting and web site control panel). Links:
Webmin Installation and Configuration in Ubuntu Linux.
FAX Server
To do
SIP/VoIP/PBX Gateway
To configure this server as VoIP gateway, I choose the well known Asterisk driven by FreePBX as a web interface. I took instructions from the tutorial
Installing freePBX on Ubuntu Server Intrepid:
# aptitude install asterisk asterisk-mysql asterisk-sounds-extra asterisk-mp3 php-db php5-gd php-pear sox curl
# adduser www-data asterisk
# chown www-data.asterisk -R /usr/share/asterisk
# usermod -s /bin/bash asterisk
In
/usr/sbin/safe_asterisk, change the variable BACKGROUND (which is 0) to 1:
BACKGROUND=1
# cd /tmp
# tar xvfz /tmp/freepbx-2.5.1.tar.gz
# cd freepbx-2.5.1/
# mysqladmin create asterisk -p
# mysqladmin create asteriskcdrdb -p
Replace MySQL root password instead of *****:
# mysql --user=root --password=***** asterisk
# mysql --user=root --password=***** asteriskcdrdb
# mysql -u root -p
mysql> GRANT ALL PRIVILEGES ON asterisk.* TO asteriskuser@localhost IDENTIFIED BY 'amp109';
mysql> GRANT ALL PRIVILEGES ON asteriskcdrdb.* TO asteriskuser@localhost IDENTIFIED BY 'amp109';
mysql> flush privileges;
mysql> quit
# cp /etc/asterisk/modules.conf /etc/asterisk/modules.conf.orig
# ./install_amp
* choose default settings
Edit the configuration file in Apache for your site (
/etc/apache2/sites-available/yourdomain.com) accordingly:
Options Indexes FollowSymLinks MultiViews
Order allow,deny
AllowOverride All
Allow from all
AuthType Basic
AuthName "Restricted Area"
AuthUserFile freepbx-passwd
Require user admin
# htpasswd -c /etc/apache2/freepbx-passwd admin
# /etc/init.d/apache2 restart
# cp /etc/asterisk/modules.conf.orig /etc/asterisk/modules.conf
To make it start at the end of everything, edit the
/etc/rc.local file before the line
exit 0.
/usr/local/sbin/amportal start
exit 0
(Optional) Asterisk will start on its own after package installation. If you want to run it under
safe_asterisk and managed by
amportal, remove asterisk from starting on its own
# update-rc.d -f asterisk remove
Add a symlink and change permissions to make your System Recordings available to IVRs.
# ln -s /var/lib/asterisk/sounds/custom /usr/local/share/asterisk/sounds/
# chown -R asterisk:asterisk /usr/local/share/asterisk/
# chmod -R 755 /usr/local/share/asterisk/
Final steps:
# chown -R asterisk:asterisk /usr/share/asterisk
# /etc/init.d/apache2 restart
SIP/VoIP/PBX server -
Asterisk. Links:
TrixBox - web interface for Asterisk,
Ekiga/GnomeMeeting (
default softphone in Ubuntu).
Other services
...
File systems & sharing
For Linux-Windows machines, and for better security (user password, file permission) - samba is best. For linux-linux machine, and for faster transfer - NFS is the way. Display directories and files is also faster on NFS.
Sharing files (Samba server)
# aptitude install samba swatFor configuration I used a
HowToForge tutorial. For details and security,
Samba documentation should be the next step. Reboot is required to use Swat (web administration tool). After reboot, open
http://server_name:901. More info in
official documentation or other links:
Samba Standalone Server With tdbsam Backend.
Alternative:
- NFS - Links:
Setting Up An NFS Server And Client On Debian Lenny.
NTFS support
Debian does not mount automatically NTFS drives (Ubuntu does!). However, NTFS support is built into linux kernel. As such, using
ntfsmount (part of
ntfsprogs) , NTFS partitions can be easily mounted and used, at command prompt:
# mkdir /mount/share/multimedia
# ntfsmount /dev/sda1 /mount/share/multimedia
To have the partitions automatically mounted upon reboot, the following line should be placed in
/etc/fstab:
/dev/sda1 /mount/share/multimedia fuse.ntfs locale=en_US.utf8 0 0
Other links of interest:
Ntfs-config,
Ntfs-3G,
NTFS vs. Ntfs-3G.
Switching file system from NTFS to Ext3
The hard drives I have for storage are formated using NTFS file system and such they were used from within Windows. I moved the drives to the new linux server and the next step is to change their NTFS file system to ext3, for safer work (NTFS under linux is not accessed same way as under Windows). To convert a drive, the following needs to be done:
>> see all available drives mounted
# df -h
>> unmount the drive
# umount /mount/shares/windows
>> delete partitions and create a linux partition of type 83 (I use only one partition on a drive)
# cfdisk /dev/sda
>> update /etc/fstab with the new files system
# vi /etc/fstab
>> this is how it should look like an ext3 mapped drive:
/dev/sda1 /mount/shares/audio/ ext3 defaults 0 0
>> format the partition with ext3
# mkfs.ext3 -b 4096 /dev/sda1
>> mount the new partition
# mount -t ext3 /dev/sda1 /mount/shares/windows
Upon formating, linux automatically reserve 5% for
root (logging etc.), which is too much. I drop it to 1 GB like this:
# tune2fs -r 108 /dev/sda1
where 108 is the number of blocks (= 1 GB). I found this info reading
a Whirlpool forum.
Mounting FAT32 devices (such as memory stick, phone etc.)
Syslog (
/var/log/syslog) shows this error:
FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
Solution: it's still under discussion, but this error shouldn't be treated as an issue. Some folks talk about this in
#483781 (Debian),
#62321 (Ubuntu),
Ubuntu Forums,
Ubuntu lists,
#126641 (RedHat), kernel lists (
message 1 and
message 2),
Google Groups etc. A temporary fix (until will be decided) is to manually mount FAT32 partition taking advices from discussion lists:
advise 1 or
advise 2.
To do: Auto-mounting supported devices
It would be good if eSata drive is "automagically" mounted (upon attach/detach), but this is not a priority now and I'll look at it later. Links I found useful:
Partitioning and Formatting Second Hard Drive - (ext3).
CD/DVD drive should not be "automagically" mounted, as this would result in busted recording (as per
dvd+rw-tools/growisofs documentation). Several auto-mounting programs, with bad results, are given as example: autofs (available in Debian repository), supermount, subfs/submount, magicdev, autorun.
Photo Albums (Gallery2)
To do
Links:
Debian Manual HowTo.
Backup
- live backup of itself or other servers (
CDP). Links:
Linux Hot Copy.
- backup of important data (compressed)
To do.
Links:
Debian Manual HowTo - Mount Windows Share.
Virtualization (hosted hypervisors)
The
hardware used on this server does not specifically support
full virtualization (call it either
hardware virtualization, or
native virtualization), but this is not a reason to avoid using it, as we'll see. As such, we'll make use of
paravitualization (call it either
software virtualization) - Debian Linux host having other operating systems as guests, managed by software which is able to run virtual machines without specifically need of
processor extensions for virtualization.
CPUs supporting virtualization natively (processor extensions) are the following (and above): Athlon 64/Opteron (AMD-V), Pentium 4/Pentium D/Multi-core (Intel VT), Xeon (Intel VT-x). The competition is hard and a lot of software projects are developed these days. RedHat develops
VMM (Virtual Machine Manager). Wikipedia has a
list of platform virtual machines (virtualization software). Some applications requires X server installed (see
minimum X.org installation).
Links:
Creating Virtual Machines For Xen, KVM, VMware Workstation 6, and VMware Server With vmbuilder On Ubuntu 8.10.
The following applications will be tested/used on this server:
VMware
VMware has
many applications for virtualization, notable the following:
-
Player version (freeware) - run (but not create) virtual machines. Use any virtual machine created by VMware Workstation, VMware Fusion, VMware Server or VMware ESX, as well as Microsoft Virtual Server virtual machines and Microsoft Virtual PC virtual machines. Import third party images including Symantec Backup Exec System Recovery (formerly called Live State Recovery) images, Norton Ghost 10 images, Norton Save & Restore images, StorageCraft ShadowProtect images, and Acronis True Image images to VMware Player compatible virtual machines.
-
Workstation version (30-days evaluation) is very flexible, but still with limitations (NTP should not run, as stated by an
Wikipedia article)
- GSX server is an entry-level virtualization server which runs virtual machines created by VMware products, as well as Microsoft Virtual PC.
-
ESX / Server version (x86) and its reduced version
ESXi (x64), both freeware, are enterprise-level virtualization server and deliver greater performance than GSX Server due to lower system overhead. Both run on
vmkernel, a customized linux kernel, which in fact is a
microkernel. ESXi has the Service Console is removed, and replaced with a minimal BusyBox installation. Disk space requirements are much lower than for ESX and the memory footprint is reduced. ESXi is intended to be run from flash disks in servers but can be run from normal disks. VMware ESXi hosts can't be managed directly from the console, all management is performed through a
VirtualCenter Server.
-
vSphere (60-days evaluation) is the industry’s first
cloud operating system. It is the next evolutionary step in IT computing, enabling customers to bring the power of cloud computing to their IT infrastructures.
Link(s):
VMware Server On Debian,
How To Install VMware Server 2 On Debian Lenny.
KQEMU (QEMU Accelerator, KDE GUI For QEMU)
KQEMU is based on
QEMU - a processor emulator (other devices emulated as well: BIOS, CD/DVD/ISO, floppy, graphics, network, serial + parallel port, IDE+PCI+ISA+USB+PS/2, sound-card, speaker). KQEMU can execute code from many guest OSes even if the host CPU does not support hardware virtualization, and supports both x86 and x86_64 CPUs. Other projects makes use of QEMU: VirtualBox, Xen-HVM, KVM (Kernel-based Virtual Machine), Win4Lin Pro Desktop
To do
Links:
QEMU Accelerator User Documentation.
VirtualBox (Innotek) / xVM (Sun Microsystems)
VirtualBox runs various versions of guest operating systems, such as: DragonFlyBSD, FreeBSD, Linux, OpenBSD, OS/2 Warp, Windows (including Windows 7), Solaris, Haiku, Syllable, ReactOS and SkyOS.
VBoxWeb (VirtualBox Web Console) allows to easily access and control VirtualBox instances remotely via web (using AJAX).
Links:
VBoxHeadless - Running Virtual Machines With VirtualBox 2 On A Headless Debian Lenny Server.
To do.
Plex86
Plex86 is an extensible free PC virtualization software program which lets PC and workstation users run multiple operating systems concurrently on the same machine. It is THE opensource free-software alternative for VMWare, VirtualPC, and other IA-32 on IA-32 "Virtual PC products."
Note: If you want to run IA-32 on a non-IA-32 architecture, then you should check out the
bochs project.
Other interesting applications:
- Adeos (Adaptive Domain Environment for Operating Systems) - running more kernels at the same time, thus allowing to run multiple operating systems, or multiple instances of a single OS
-
Bochs - open source IA-32 (x86) PC emulator written in C++.
-
coLinux (Cooperative Linux) - (open-source) software which allows Microsoft Windows and the Linux kernel to run simultaneously in parallel on the same machine. In contrast to traditional VMs, the CVM shares resources that already exist in the host OS.
-
Debootstrap - allows to create a Debian base system from scratch, without requiring the availability of dpkg or apt. It does this by downloading .deb files from a mirror site, and carefully unpacking them into a directory which can eventually be chrooted into (using
pbuilder). Another implementation of the same concept is
cdebootstrap (C implementation of Debootstrap). Worth reading:
Testing cdebootstrap and debootstrap (message board),
Create a Debian VM with debootstrap,
HOWTO: Bootstrapping Debian Linux System using debootstrap and chroot.
-
DOSBox - emulates an IBM PC compatible computer running MS-DOS.
-
FreeVPS (+ H-Sphere, an automated scalable web hosting software) - a cost effective solution that allows running many virtually isolated standalone servers on one host box which extends the vserver solution with a series of improvements.
-
JPC (emulator) - x86 emulator written in pure Java which can run on any platform that supports the Java Virtual Machine as a virtual PC compatible machine that can run MS-DOS and other x86 operating systems. Programs inside JPC can run up to 20% of the native processor speed. It is nice that can run in a web browser (I guess :-) ).
-
KVM (Kernel-based Virtual Machine) - full virtualization solution (open-source) on x86 hardware containing virtualization extensions (Intel VT or AMD-V), similar in functionality with Xen, QEMU etc. (KVM also requires a modified QEMU, although work is underway to get the required changes upstream.). A wide variety of guest operating systems work with KVM, including many flavours of Linux, BSD, Solaris, Windows, Haiku, ReactOS and AROS Research Operating System. By itself, KVM does not perform any emulation. Instead, a user-space program uses the /dev/kvm interface to set up the guest VM's address space, feed it simulated I/O and map its video display back onto the host's.
-
Linux-VServer - containers-based, provides virtualization for GNU/Linux systems using kernel level isolation (processes run on the same kernel), thus only linux guests can run which share the kernel. It is similar to: OpenVZ, Parallels Virtuozzo Containers, the FreeBSD jail mechanism, iCore Virtual Accounts, Solaris Containers, FreeVPS (an early fork of Linux-VServer).
- Parallels' variants of commercial applications (none free), based on OpenVZ: Parallels Workstation (50€,
some imitations),
Parallels Desktop (70€),
Parallels Workstation Extreme ($400 per machine / $250 per pop, it can run dedicated graphics for virtualized environments),
Parallels Server (Beta/free, as of this writing),
Parallels Virtuozzo Containers ($2500).
-
OpenVZ - containers-based, allows a physical server to run multiple isolated operating system instances (same kernel as the host), having only a 1–3% performance penalty as compared to using a standalone server
-
PearPC - an PowerPC platform emulator capable of running many PowerPC operating systems, including Mac OS X, Darwin and Linux.
-
UML (User-mode Linux) - enables multiple virtual Linux systems ("guests") to run as an application within a normal Linux system ("host"). In UML environments, host and guest kernel versions don't need to match, as such different kernels can be used.
-
Win4Lin ($30-Ubuntu/$50-Others) - a proprietary software application which allows users to run a copy of Microsoft Windows 95, 98, Me, 2000 or XP application on their desktop. Win4Lin is designed with business users in mind, and as such, does not support features such as MIDI, in favor of support for Microsoft Office-style application compatibility
-
Xen - full virtualization solution (open-source) structured with the Xen hypervisor as the lowest and most privileged layer. The first guest operating system - "domain 0" (dom0), is booted automatically when the
hypervisor boots and given special management privileges and direct access to all physical hardware by default. The system administrator can log into dom0 in order to manage any further guest operating systems, called "domain U" (domU). Modified versions of Linux, NetBSD and Solaris can be used as the dom0. On certain hardware, as of Xen version 3.0, unmodified versions of Microsoft Windows and other proprietary operating systems can also be used as guests if the CPU supports x86 virtualization (e.g., Intel VT or AMD-V). Xen can be delivered to market as a virtualization platform, such as Citrix XenServer Enterprise Edition (formerly XenSource's XenEnterprise), or embedded within the host operating system. On most CPUs, Xen uses
paravirtualization. Through paravirtualization, Xen can achieve high performance even on its host architecture (x86) which is notoriously uncooperative with traditional virtualization techniques. On x86, the Xen host kernel code runs in
Ring 0, while the hosted domains run in
Ring 1 or
Ring 3. Xen host operates in root mode and has access to the real hardware, while the unmodified guest operates in
Rings 0-3 of non-root mode and its "hardware" accesses are under complete control of the hypervisor. Xen-HVM has device emulation based on the QEMU project to provide I/O virtualization to the VMs. Hardware is emulated via a patched QEMU "device manager" (qemu-dm) daemon running as a backend in dom0. This means that the virtualized machines see as hardware: a PIIX3 IDE (with some rudimentary PIIX4 capabilities), Cirrus Logic or vanilla VGA emulated video, RTL8139 or NE2000 network emulation, PAE, and somewhat limited ACPI and APIC support and no SCSI emulation. Xen virtual machines can be "live migrated" between physical hosts across a LAN without loss of availability, with a penalty of 60–300 ms required to perform final synchronization. Xen under Linux currently runs on x86, with Pentium II or newer processors, x86-64 based systems, as well as on IA-64 and PowerPC. Xen supports up to 64-way symmetric multiprocessing machines. Debian includes Xen 3.2.1 in its stable release 5.0 (Lenny). Guest systems can run fully virtualized (requires special hardware) or paravirtualized (requires guest OS code modification). On the list of the supported systems patched to operate as a paravirtualized Xen guest, are: Linux (paravirtualization integrated in 2.6.23, patches for other versions exist), Minix, NetBSD (NetBSD 2.0 has support for Xen 1.2, NetBSD 3.0 has support for Xen 2.0, and NetBSD 3.1 supports Xen 3.0), OpenBSD (announced here but discontinued), FreeBSD (Limited, experimental support for Xen 3 in 8-CURRENT), OpenSolaris, NetWare, Microsoft Windows (unmodified, if the processor supports hardware virtualization provided by Intel VT or AMD-V).
Links:
Debian Wiki - Xen,
Debian Lenny xen server setup,
Creating A Fully Encrypted Para-Virtualised Xen Guest System Using Debian Lenny,
Comparison of platform virtual machines,
Technical comparison of Linux virtualization technologies,
Debian virtualization (Google search),
How To Compile virt-df, virt-top, virt-mem & virt-ctrl On Debian Lenny.
Web monitoring
To do
Links:
Debian Manual HowTo - AWStats on Debian.
Wifi support
Debian installs
ath5k_pci wifi driver for my card. This is what I get in Debian:
# dmesg | grep ath
[ 10.969577] ath5k_pci 0000:05:00.0: registered as 'phy0'
[ 11.131604] ath5k phy0: Atheros AR2414 chip found (MAC:0x79,PHY: 0x45)
# lspci | grep Atheros
05:00.0 Ethernet controller: Atheros Communications Inc. AR5212/AR5213 Multiprotocol MAC/baseband processor (rev 01)
... and this is what I get in Ubuntu Hardy Live CD:
# dmesg | grep ath
[ 89.685905] ath_hal: module license 'Proprietary' taints kernel.
[ 89.867822] ath_hal: 0.9.18.0 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
[ 90.171804] ath_pci: 0.9.4
[ 91.021601] ath_rate_sample: 1.2 (0.9.4)
[ 137.045049] ath0: no IPv6 routers present
# lspci | grep Atheros
05:00.0 Ethernet controller: Atheros Communications Inc. AR5212/AR5213 Multiprotocol MAC/baseband processor (rev 01)
I looks like Ubuntu 8 (Hardy Heron) Live CD installs
ath_hal driver for my D-Link G-520 PCI card.
I followed this
tutorial, but not very satisfied of the result - I could not make it take automatically IP address from DHCP, connecting to a secured wireless router. Of great help could be the following links:
Debian Manual HowTo - Authenticate with wpa using PSK TKIP,
Installing MadWifi 'By Hand',
Debian specific docs @MadWifi,
Wireless Tools for Linux (not installed by default, but required for commands like
iwconfig,
iwlist etc.),
Debian Wireless Fidelity (wifi wiki),
Atheros wireless devices (ath5k),
Atheros AR5xxx devices (MadWifi),
WPA support in Debian,
Linux Wireless,
wmaster0/wifi0 master device (Ubuntu thread),
Atheros AR5007EG/AR242x wireless cards in Ubuntu 8.10 (Intrepid Ibex),
ThinkWiki,
WiFi and Debian,
How To Connect To A WPA Wifi Using Command Lines On Debian.
Setting it up in Master Mode... not yet done!
Links:
Pat Erley' work (using
hostapd and
mac80211 Linux API),
Linux for Internet Providers,
Multiband Atheros Driver for WiFi (madwifi) package for Debian,
Madwifi HOWTO - FAQ - WIKI, Wifi Access Point with hostap + hostapd + freeradius + mysql backend:
Part 1 and
Part 2,
WPA2 access point under GNU/Linux.
Torrents
[?] I gave a shoot to
rtorrent, which I find it nice:
# aptitude install rtorrent
Starting the application, an error appear:
Could not read resource file: ~/.rtorrent.rc
That is easily fixable with this command:
# cp /usr/share/doc/rtorrent/examples/rtorrent.rc ~/.rtorrent.rc
To add rtorrent to startup:
# wget http://libtorrent.rakshasa.no/attachment/wiki/RTorrentCommonTasks/rtorrentInit.sh
# mv rtorrentInit.sh /etc/init.d/rtorrent
# update-rc.d rtorrent defaults
Add web interface for remote control (rtGui):
# aptitude install php5-xmlrpc libapache2-mod-scgi
After installation of required packages, I followed
this tutorial for configuration.
Links: man page,
Headless torrent downloads with rTorrent and Screen,
Compiling and Installing rTorrent with LibTorrent on Ubuntu/Debian,
Common Tasks in rTorrent for Dummies,
How to Install the latest rtorrent and libtorrent (from source),
rtorrent with wtorrent on debian etch complete (w. screenshoots).
Web interface: RTPG (Rtorrent Perl GUI) -
tutorial,
rtGui (PHP/XML-RPC, Ajax),
wTorrent (xmlrpc/Ajax),
rTWi (PHP),
nTorrent (graphical user interface client to rtorrent, written in Java),
n2hell - Ajax browser UI for rtorrent (not available in Debian repository),
TorrentFlux (web interface,
working great with Transmission, but with other clients too: BitTornado, Mainline),
Torrentflux-b4rt (web based transfer control client; requires database),
Installing Torrentflux-b4rt on Ubuntu/Debian,
Torrentflux B4rt on Ubuntu Hardy,
Torrentflux-b4rt 1.0 README.
[?] Transmission
[Try 1]
Transmission in Lenny's main repository is
rather old (1.22-1), thus we need to use a
newer version (1.74-1). Make sure backports repository is installed. If yes, the rest is easy:
(0)
Build transmission-daemon .deb package, in order to create ALL its required data
(1) Grab and install Transmission from backports:
# aptitude -t lenny-backports install transmission
(2) Create a user "transmission" with blank password:
# adduser --disabled-password transmission
(3) Create a init.d script to run at startup having the content from
Transmission website:
# vim /etc/init.d/transmission-daemon
(4) Set correct permissions:
# chmod +x /etc/init.d/transmission-daemon
# chown root:root /etc/init.d/transmission-daemon
(5) Start the daemon:
# /etc/init.d/transmission-daemon start
I'm working on making Transmission works from a Windows machine, just like
uTorrent (only that the downloads are saved on linux server, not on Windows machine). There is
transmission-remote-dotnet client for Windows, but I didn't yet succeeded to make it work. Probably because step 0 is not completed...
[Try 2]
(1) Edit
/etc/apt/apt.sources and add
Sid (unstable) repository:
deb http://ftp.ro.debian.org/debian/ sid main contrib non-free testing unstable
(2) Update local repository:
# aptitude update
(3) Install Transmission (1.74.8994, as of this time):
# aptitude install transmission transmission-daemon *
* A warning appear that old version of transmission will be erased and new version installed, including dependencies (erased/re-installed as well)
(4) Transmission may already be started, we'll stop it to edit config file:
# /etc/init.d/transmission-daemon stop
(5) Edit configuration
/etc/transmission-daemon/settings.json, pay attention to the following:- download directory >> choose your preferred, if you like:
"download-dir": "\/var\/lib\/transmission-daemon\/downloads",
- choose a password and enter instead of the default one (random choose by default, as you see bellow):
"rpc-password": "{ee3da850ac90491cd6579e33b3f43ba17d6cbaf6Y9Mxh0k3",
- add your IP to "white list":
"rpc-whitelist": "127.0.0.1,192.168.*.*",
(6) Start Transmission:
# /etc/init.d/transmission-daemon start
(7) Check that it works, type server's IP in your browser - it will ask for user (transmission) and password (what you typed in config)
(8) Remember to remove
Sid (unstable) repository from
/etc/apt/apt.sources:
Everything else should be self-explanatory...
Hmm... Torrents not working. 'Though I managed to install succesfully and make Transmission Remote work, torrents do not download files. Error log of Transmission Remote tells:
No such file or directory (/path/to_torrent)
I don't have any clue...
Tips:
>> to view status statistics at the console (and daemon version) {--session-stats}:
# transmission-remote -n user:pass -st {username "transmission" was set before, in tutorial}
>> to view session details at the console (and daemon version) {--session-info}:
# transmission-remote -n user:pass -si {username "transmission" was set before, in tutorial}
>> to show list of torrents at the console {--list}:
# transmission-remote -n user:pass -l {username "transmission" was set before, in tutorial}
Other clients: bittorrent (the original client; it has an CLI interface),
deluge (client, web interface),
ctorrent,
Enhanced CTorrent, ktorrent - crashing; gui + web interface,
ABC [Yet Another Bittorrent Client] - client gui and web interface,
BitTorrent client BTG and its Web user interface wwwBTG on Debian 4.0 Etch.
Other links:
How to Use BitTorrent in Linux,
Updated dns-323 bt download management scripts.
To do (reminder for myself)
- KVM switch over IP:
KVM Switches For the Home and the Enterprise - (Avocent).
-
WebCam under Linux,
Webcam on debian,
Motion - a software motion detector,
CLI Magic: Getting into Motion,
webcam-server package.
- print server for local network (using CUPS); links:
Debian and Windows Shared Printing mini-HOWTO (2005),
A Brief Introduction to Network Printing with CUPS (2005),
Securing printing access (5.5),
Printing HOWTO by Grant Taylor & Dirk Allaert (2003)
-
Nullmailer (5.6.1) configuration for managed systems
- check other
crontab jobs:
find `find /etc/ -type d -iname cron\*` -type f -o -type l && grep -v ^# /etc/crontab && awk -F':' '{print $1}' /etc/passwd | xargs -iU crontab -l -u 'U' 2>&1| grep -v ^no
- organize logs for easier reading
- remove
dmesg from
/var/log/messages
- sync time between BIOS clock and updated OS clock regularly
- (transparent) proxy/cache server (
squid - links:
Securing Debian Manual,
How to Setup Transparent Squid Proxy Server in Ubuntu)
- auto-update OS, antivirus, anti-spam
- hardware inventory; link(s):
Install GLPI (IT and asset Managemet Software) from Ubuntu Repositories.
- traffic control:
Linux Advanced Routing & Traffic Control HOWTO.
- ftp using virtual users (same table as mail users); vsftp is a good choice;
setup link;
security should not be forgot
- few programs to keep an eye on
- SoX (Sound eXchange) - Swiss army knife of sound processing (quick howto)
- checkmp3 - Identify MP3s that do not follow the MP3 format
- mp3check - tool to check mp3 files for consistency
- mp3blaster - Full-screen console mp3 and Ogg Vorbis player (ncurses)
- mp3report - Script to create an HTML report of MP3 files in a directory
- mplayer - media player (can be used in console)
- ncmpc - front-end for mpd (Music Player Daemon)
- gamp -
- multimedia apps
- hmp3
- apps using mad
-
About / Despre acest blog
Disclaimer and privacy statement / Confidenţialitate
Updated / Actualizat: 2009-09-22.
___
