Principle 1: Automating best practice

A good systems administrator knows what tasks must be done, either regularly or occasionally, to manage an Internet-connected server. Some tasks, such as backups and ensuring system security, are regular and ongoing. Other tasks, such as setting up file sharing or adding a new user, are only performed from time to time. In each case a good system administrator will not only know how to do the task itself, but also how to do it in a secure, maintainable, extensible and efficient manner, in accordance with current industry best practice.

However, not every server has a dedicated, experienced system administrator. This is especially the case in small businesses, where there may be no IT staff at all. Therefore, the goal of the SME Server is to automate the activities performed by a good sysadmin, from simple tasks such as adding users right through to backups and other complex activities, to the point where they can be easily performed by someone with little or no technical knowledge of the system.

Principle 2: Simplicity

The SME Server is characterized by its tight focus on providing network server functions. For the end-user, the SME Server provides simple, extensible web-based management. For developers, the SME Server provides clean, consistent, extensible interfaces to allow features to be added and modified.

Some Linux distributions are aimed at desktop users, general purpose server applications, or "enterprise" server applications. The SME Server is different in that it is targeted towards providing network server functionality for small to medium enterprises. Because of this, the SME Server is much smaller than many other Linux distributions, as software packages which are not needed for this purpose (for example, the X window system) are not included in the distribution.

The SME Server is also simple for a non-technical person to manage. For an end-user administering the server, choices are kept to a minimum. If a decision is very likely to be the same for all small businesses, the answer is assumed and the end user is not required to make a choice. When decisions are required, they are phrased in terms independent of the underlying technology, so that end-users are not required to be intimately familiar with Linux or Linux applications.

For developers, the simplicity is in the architecture of the SME Server system. Features are layered in such a way that additional features can be added without affecting the current services, and often without requiring modifications to the user interface.

Principle 3: Extensibility

SME Server's third design goal is extensibility, which provides a balance to the simplicity previously described. Since the simplest possible server will not suit every need, we make it easy to customize and extend the server in a number of ways.

Firstly, interfaces are provided for experienced users to customize the system from the Linux command line. These include tools to manipulate the configuration database, trigger events, or modify the configuration files for the various software installed on the system.

Secondly, applications allows developers to create additional software modules which can be easily installed and configured by end-users. Applications may provide application software for the server's users, administration tools, network services, or any other type of software or data.

The SME Server architecture explicitly supports developers by making it easy to drop software into place and remove it without needing to modify existing files. For instance, a web application does not need to edit the web server configuration file, but can simply drop a template fragment into the appropriate directory on the system and be assured that it will be expanded into the configuration file as required.

Principle 4: Reliability

The SME Server is designed to run without intervention 24 hours per day, seven days a week. This reliability has been designed from the ground up: stable, well supported versions of the Linux kernel and applications, RAID disk mirroring, automatic firewall, and process supervision. Where applications have been shown to be insecure or unreliable, we use stable, secure replacements. The modular architecture allows this to be done without affecting the system administrator's view of the system and with only localised effect on the developer's view.

Overview

All user-modifiable configuration parameters on the SME Server are stored in the configuration database. These values are used to generate the system configuration files, such as those found in the /etc/ directory.

The configuration databases may be modified by various programs on the system, including the SME Server manager, the SME Server console, or scripts run from the command line by a system administrator.

Each entry in the database is either a simple key/value pair or a key and a collection of related property/value pairs.

[root@gsxdev1 ~]# config show AccessType
AccessType=dedicated

[root@gsxdev1 ~]# config show ConsoleMode
ConsoleMode=login

[root@gsxdev1 ~]# config show TimeZone
TimeZone=Australia/NSW

[root@gsxdev1 ~]# config show atalk
atalk=service
    MaxClients=20
    status=enabled

[root@gsxdev1 ~]# config show dhcpd
dhcpd=service
    end=192.168.1.250
    start=192.168.1.65
    status=disabled

[root@gsxdev1 ~]# config show LocalIP
LocalIP=192.168.1.100

[root@gsxdev1 ~]# db configuration show LocalIP
LocalIP=192.168.1.100

[root@gsxdev1 ~]# db accounts show admin
admin=system
    EmailForward=local
    FirstName=Local
    ForwardAddress=
    LastName=Administrator
    Lockable=no
    PasswordSet=yes
    Removable=no
    Shell=/sbin/e-smith/console
    VPNClientAccess=no

[root@gsxdev1 ~]# db
usage:
    /sbin/e-smith/db dbfile keys
    /sbin/e-smith/db dbfile print [key]
    /sbin/e-smith/db dbfile show [key]
    /sbin/e-smith/db dbfile get key
    /sbin/e-smith/db dbfile set key type [prop1 val1] [prop2 val2] ...
    /sbin/e-smith/db dbfile setdefault key type [prop1 val1] [prop2 val2] ...
    /sbin/e-smith/db dbfile delete key
    /sbin/e-smith/db dbfile printtype [key]
    /sbin/e-smith/db dbfile gettype key
    /sbin/e-smith/db dbfile settype key type
    /sbin/e-smith/db dbfile printprop key [prop1] [prop2] [prop3] ...
    /sbin/e-smith/db dbfile getprop key prop
    /sbin/e-smith/db dbfile setprop key prop1 val1 [prop2 val2] [prop3 val3] ...
    /sbin/e-smith/db dbfile delprop key prop1 [prop2] [prop3] ...

use esmith::AccountsDB;

my $db = esmith::AccountsDB->open or die "Couldn't open AccountsDB\n";

my $admin = $db->get("admin") or die "admin account missing from AccountsDB\n";

print $admin->show();

admin
     EmailForward = local
        FirstName = Local
   ForwardAddress =
         LastName = Administrator
         Lockable = no
      PasswordSet = yes
        Removable = no
            Shell = /sbin/e-smith/console
  VPNClientAccess = no
             type = system

perldoc esmith::ConfigDB
perldoc esmith::AccountsDB
perldoc esmith::HostsDB
perldoc esmith::NetworksDB

perldoc esmith::DB

[root@gsxdev1 db]# cd /etc/e-smith/db
[root@gsxdev1 db]# ls
accounts       domains       networks       yum_installed
backups        hosts         spamassassin   yum_repositories
configuration  mailpatterns  yum_available  yum_updates

[root@gsxdev1 db]# ls configuration/
defaults  force  migrate

[root@gsxdev1 db]# cat configuration/defaults/sshd/status
disabled

[root@gsxdev1 db]# cat configuration/force/sysconfig/ReleaseVersion
7.0rc2

{
    my $popd = $DB->get("popd") or return;

    my $pop3 = $DB->get("pop3") ||
        $DB->new_record("pop3", { type => "service" });

    $pop3->merge_props($popd->props);

    $popd->delete;
}

The configuration databases

Namespace issues

All entries in a single database share the same namespace. Users, groups, information bays, printers, and other entries in the accounts database currently all share one namespace. This means that you cannot have a user with the same name as an information bay, group or other entry in the accounts database.

However, it would be possible to have a host named fredfrog as well as a user named fredfrog as they are stored in separate databases and thus different namespaces.

Actions

An action is a program, frequently written in a scripting language, which performs a single task. It is typically an encapsulation of a task usually done by a system administrator, such as editing a configuration file or reconfiguring a service. Actions are not called directly; they are always called by signalling an event.

The actions are stored in the /etc/e-smith/events/actions/ directory. These actions are then linked into the relevant events as the same action may need to be performed in more than one event.. To create a new action called myaction you simply create a program to perform the action myaction and save it as /etc/e-smith/events/actions/myaction . Actions can be written in any programming language, although additional platform support is provided for Perl code.

An example action script is set-external-ip which is called when the external IP address changes. Here's the body of that script (at time of writing):

package esmith;

use strict;
use Errno;
use esmith::ConfigDB;

my $db = esmith::ConfigDB->open or die "Couldn't open ConfigDB\n";

my $event = $ARGV[0];
my $newip = $ARGV[1];

$db->set_value('ExternalIP', $newip);
$db->set_prop('ExternalInterface', 'IPAddress', $newip);

exit 0;

This script sets the ExternalIP value and the IPAddress property of the ExternalInterface record in the configuration database to the value provided as a parameter. The $event parameter is not used in this particular script.

Events

Events are a mechanism which allows the system to trigger a set of actions in response to actual events that happen on the system. When one of the users interfaces modifies the configuration databases, it must signal an event to regenerate the various server application configuration files according to the new configuration. The user interface must never modify configuration files directly.

Each event is associated with a list of actions which should be performed when that event occurs and is defined as a subdirectory of /etc/e-smith/events/ containing symbolic links to the appropriate actions, loosely modelled after the System V init mechanism for starting servers. For example, if you examine the /etc/e-smith/events/ip-change directory:

lrwxrwxrwx  1 root root   26  S15set-external-ip -> ../actions/set-external-ip*
lrwxrwxrwx  1 root root   21  S85update-dns -> ../actions/update-dns*
drwxr-xr-x  2 root root 4096  services2adjust/
drwxr-xr-x  5 root root 4096  templates2expand/

The symbolic links are given prefixes such as S15, S85, etc. to specify the order in which the actions should be executed in a similar manner to the System V init mechanism.

You can change the actions performed by an event by changing the links in the event directory. You can also create a new event by creating another subdirectory of /etc/e-smith/events/.

lrwxrwxrwx  1 root root 6  masq -> adjust
lrwxrwxrwx  1 root root 7  ntpd -> restart
lrwxrwxrwx  1 root root 7  pptpd -> sigterm
lrwxrwxrwx  1 root root 6  qmail -> sighup
lrwxrwxrwx  1 root root 7  tinydns -> sigusr2

signal-event console-save

Feb  2 13:22:33 gsxdev1 esmith::event[4525]: 
  S65sshd-conf=action|
  Event|remoteaccess-update|
  Action|S65sshd-conf|
  Start|1138846952 730480|
  End|1138846953 66768|
  Elapsed|0.336288

Feb  2 16:11:54 gsxdev1 esmith::event[4787]: 
  S99false=action|
  Event|remoteaccess-update|
  Action|S99false|
  Start|1138857114 58910|
  End|1138857114 81920|
  Elapsed|0.02301|
  Status|256

Design of the template system

Every piece of software has its own configuration format, and writing parsers for each one is a complex, time-consuming and error-prone process. The SME Server software avoids the whole issue by using templates which generate the correct configuration.

In most cases, SME Server configuration files are over-written when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file (e.g. /etc/fstab) is also automatically updated by some other process.

Templates are stored under /etc/e-smith/templates/ in a directory hierarchy which matches the standard filesystem. For example, the template for /etc/inittab is stored in the /etc/e-smith/templates/etc/inittab/ directory. Each template is stored as a directory of template fragments and processed by the Perl Text::Template module.

The template fragments are concatenated together in ASCIIbetical order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of the SME Server's modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.

The Text::Template module

The Text::Template module allows arbitary Perl code to be embedded in a template file by surrounding it in braces ("{" and "}"). The code inside the braces is interpreted and its return value replaces the section between, and including, the braces. For instance:

The answer is { 2 + 2 }

becomes

The answer is 4

Variables can be passed in from the program which is expanding the template, hence:

Shopping list:
{ 
    $OUT = '';

    for my $item ( qw(bread milk bananas) )
    {
        $OUT .= "* $item\n";
    }
}

would expand to:

Shopping list:
* bread
* milk
* bananas

The SME Server template system uses this mechanism to automatically pass in global configuration variables from the configuration database which can then be used to fill out the configuration files.

For example, the /etc/hosts template is fairly simple and composed of two fragments:

[gordonr@smebuild hosts]$ pwd
/etc/e-smith/templates/etc/hosts

[gordonr@smebuild hosts]$ ls
10localhost  20hostname

Let's look at those fragments. The first is a piece of static text, which Text::Template will include verbatim:

127.0.0.1       localhost

The second is more complex and relies on values from the configuration database:

{
    $OUT .= "$LocalIP\t";
    $OUT .= " ${SystemName}.${DomainName}";
    $OUT .= " ${SystemName}";
}

Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code. When this template is expanded, it results in the following configuration file:

#------------------------------------------------------------
#              !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------

127.0.0.1       localhost
192.168.10.1    smebuild.gormand.com.au smebuild

The header block comes "for free" as part of the template system, courtesy of an optional file template-begin, which is always processed as the first fragment. If it isn't provided, the text shown with # comments is included.

The other lines are provided by the two fragments shown above. Note the use of the configuration database variables: $LocalIP, $SystemName and $DomainName. All simple entries in the configuration database are provided as global variables to the templates.

Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments.

The complex entries in the configuration database are also provided as global variables to the templates. However, they are provided as Perl hashes instead of simple scalars. For example, here is how you might configure the Network Time Protocol (NTP) server /etc/ntp.conf file:

server { $ntpd{NTPServer} }
driftfile /etc/ntp/drift
authenticate no

The NTPServer setting is stored in the ntpd configuration database record, and so can be accessed via the hash accessor $ntpd{NTPServer}.

[gordonr@sevendev1 devguide]$ ls /etc/e-smith/templates-default/
template-begin       template-begin-perl   template-end-php
template-begin-html  template-begin-php
template-begin-pam   template-begin-shell

/etc/e-smith/templates/etc/book.conf:
10intro
30chapter3
40chapter4
80synopsis

/etc/e-smith/templates-custom/etc/book.conf:
30chapter3
50chapter5

Template expansion

[gordonr@smebuild templates2expand]$ pwd
/etc/e-smith/events/ip-change/templates2expand

[gordonr@smebuild templates2expand]$ find . -type f
./etc/services
./etc/pam.d/passwd
./etc/dhcpd.conf
./etc/pptpd.conf
./etc/securetty
./etc/hosts.deny
./etc/shells
./etc/proftpd.conf
./etc/fetchmail
./etc/ppp/options.pptpd
./etc/ppp/ip-down.local
./etc/ppp/ip-up.local
./etc/hosts.allow
./etc/startmail
./var/qmail/alias/.qmail-localdelivery-default
./var/qmail/alias/.qmail-default
./var/qmail/control/concurrencylocal
./var/qmail/control/me
./var/qmail/control/virtualdomains
./var/qmail/control/smtproutes
./var/qmail/control/plusdomain
./var/qmail/control/doublebounceto
./var/qmail/control/rcpthosts
./var/qmail/control/badhelo
./var/qmail/control/databytes
./var/qmail/control/mailrules.default
./var/qmail/control/helohost
./var/qmail/control/bouncehost
./var/qmail/control/envnoathost
./var/qmail/control/defaultdomain
./var/qmail/control/locals
./var/qmail/control/bouncefrom
./var/qmail/control/defaulthost
./var/qmail/control/concurrencyremote
./home/e-smith/.qmail

[gordonr@smebuild templates2expand]$ find . -type f|xargs rpm -qf | sort | uniq
e-smith-base-4.15.6-01
e-smith-email-4.15.4-01
e-smith-pptpd-1.11.0-18
e-smith-proftpd-1.11.0-25
e-smith-qmail-1.9.0-11
smeserver-qpsmtpd-1.0.1-09

UID="root"
GID="daemon"
PERMS=0755

TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX"
OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1"
MORE_DATA={ THIS_DEVICE => "eth1" }
FILTER=sub { $_[0] =~ /^#/ ? '' : $_[0] } # Remove comments

perldoc esmith::templates

expand-template filename

use esmith::templates;

foreach my $group (@groups)
{
    my $groupName = $group->key;

    [...]

    processTemplate(
            {
                CONFREF =>
                    {
                        Members => $members,
                    },

                TEMPLATE_PATH =>
                    "/var/qmail/alias/.qmail-group",

                OUTPUT_FILENAME => "/var/qmail/alias/.qmail-$groupName",
            }
        );

    [...]
}

Process startup

In typical Linux systems, services (processes) are started at boot time through a mechanism such as System V init. When the system administrator needs to change the settings, they modify the configuration files and then restart the service or notify the process that it needs to re-read the configuration.

It is usually assumed that processes which have been started will continue to run, and only require intervention during configuration changes. There are a number of problems with this model, which are addressed by the SME Server:

• Processes do occasionally fail through software errors, memory exhaustion and accidental finger poking by the system administrator.

• Some startup scripts and processes do not gracefully handle server crashes, such as power outages. The startup scripts and processes often use process identifier (PID) files to determine whether the process is running. Reliable handling of PID files is impossible to achieve under all failure cases.

• Many processes do not deal properly with rapid invocation of stop and start requests. This is often, but not always, due to "PID file race" conditions.

Process supervision: runit (and supervise)

The SME Server addresses these issues by running processes under the runit process supervision environment, which:

• runs each process under control of its own supervisor process

• imposes process limits

• restarts the process if it fails

• provides a consistent mechanism for controlling the underlying process

[root@gsxdev1 events]# pstree 1
init-+-acpid
     |-md1_raid1
     |-md2_raid1
     | ...
     |-runsvdir-+-runsv-+-multilog
     |          |       `-ulogd
     |          |-6*[runsv---multilog]
     |          |-runsv-+-multilog
     |          |       `-ntpd
     |          |-runsv-+-multilog
     |          |       `-tinydns
     |          |-runsv-+-cvm-unix
     |          |       `-multilog
     |          |-runsv-+-multilog
     |          |       `-mysqld
     |          |-5*[runsv-+-multilog]
     |          |          `-tcpsvd]
     |          |-runsv-+-multilog
     |          |       `-oidentd
     |          |-runsv-+-multilog
     |          |       `-smtp-auth-proxy
     |          |-runsv-+-multilog
     |          |       `-smbd---smbd
     |          |-runsv---httpd---10*[httpd]

S00microcode_ctl     -> /etc/rc.d/init.d/e-smith-service
S05syslog            -> /etc/rc.d/init.d/e-smith-service
S06cpuspeed          -> /etc/rc.d/init.d/e-smith-service
S15nut               -> ../init.d/e-smith-service
S15raidmonitor       -> /etc/rc.d/init.d/e-smith-service
S26apmd              -> /etc/rc.d/init.d/e-smith-service
S35bootstrap-console -> /etc/rc.d/init.d/e-smith-service
[...]

The web directory

The primary files which make up the SME Server manager are kept in the /etc/e-smith/web/ directory. These files define the layout of the web functions and require auxiliary files which provide translations and the implementation of the functions.

Web function scripts

The functions subdirectory contains all of the screen definitions for all panels. Each screen definition is a CGI script which displays the screen and also handles the CGI form submission. The scripts are written using the CGI::FormMagick toolkit, which separates the screen layout from the panel implementation code, facilitates form validation and provides full support for localisation of the manager.

#!/usr/bin/perl -wT
# vim: ft=xml:

#----------------------------------------------------------------------
# heading     : Demo
# description : Logger
# navigation  : 1000 1000
#----------------------------------------------------------------------

use strict;
use warnings;

use esmith::FormMagick::Panel::loggerdemo;

my $f = esmith::FormMagick::Panel::loggerdemo->new();
$f->display();

<form
    title="FORM_TITLE"
    header="/etc/e-smith/web/common/head.tmpl"
    footer="/etc/e-smith/web/common/foot.tmpl">

    <entry>
        <base>FORM_TITLE</base>
        <trans>Logger demo</trans>
    </entry>

    <page name="First" pre-event="print_status_message()"
        post-event="change_settings">

        <field
            type="select"
            id="loggerdemo_Interval"
            options="10,20,30,40,50"
            value="get_interval()">
            <label>LABEL_LOGGERDEMO_INTERVAL</label>
        </field>

        <field
            type="select"
            id="loggerdemo_status"
            options="'disabled' => 'DISABLED', 'enabled' => 'ENABLED'"
            value="get_status()">
            <label>LABEL_LOGGERDEMO_STATUS</label>
        </field>

        <subroutine src="print_button('SAVE')" />

    </page>

</form>

# heading     : Configuration
# description : E-mail
# navigation  : 6000 6700

# heading     : Security
# description : Remote access
# navigation  : 5000 5200

# heading     : Security
# description : Local networks
# navigation  : 5000 5300

# heading     : Security
# description : Advanced security
# navigation  : 5000 5250

Common files

The common subdirectory contains any static files (such as images) which are used by multiple panels.

Panel definitions

The panels directory contains the panel definitions. There is one subdirectory for each panel. Each panel must have html and cgi-bin subdirectories. The cgi-bin subdirectory should contain only symbolic links to the actual CGI scripts in the functions directory, and the html directory should contain the main index.html file for the panel, as well as any required navigation links.

Keeping the CGI scripts for all panels in a shared directory makes it much easier to create auxiliary panels with slightly different options and permissions. You can just copy the entire panel directory, then customize the access permissions and navigation links. For example, it would be very straightforward to create a password-protected panel which only allowed the creation and deletion of user accounts. That task could be delegated to administrative staff.

Creating a development environment

Packages which do not require compilation, for example shell and perl scripts, can be built on the SME Server platform. All of the examples in this documentation can be performed on a standard SME Server installation.

Before attempting to compile any software, you should check whether the package is available from one of the many well-maintained RPM repositories. Using these RPMs will ensure compatibility with the other RPMs on the SME Server. You are likely to find the package you want in either the CentOS or Dag Wieers repositories.

If an RPM does not already exist, you should install a CentOS developer workstation or server for SME Server development. You will also need to install the e-smith-devtools packages which can be found on the SME Server CD.

Exercise 1: Changing a configuration template

Let us say that you wish to customize your server so that it runs a specified program every twenty minutes. To simplify the problem, let us assume that this program simply adds a line of dots to the log file (/var/log/messages), i.e.:

/usr/bin/logger -t "Demo" "......"

Normally you would accomplish this by adding a line to the /etc/crontab file, which is the standard Linux mechanism for running scheduled jobs. However, the default /etc/crontab file looks something like this on an SME Server:

#------------------------------------------------------------
#              !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

# run-parts

01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

# logrotate
12 1 */7 * * root       /sbin/e-smith/signal-event logrotate

Note the auto-generated comment block which reminds you not to edit the file. If you do, your changes will be overwritten when the template is next expanded by a system event. We want to append a new line that looks like this (read the Linux crontab documentation to understand the format of crontab entries):

*/20  *  *  *  *   root   /usr/bin/logger -t "Demo" "......"

Remember that we cannot simply edit the /etc/crontab file. The rule is that we must perform this customization by adding a new file to the system. To get an idea how to do this, have a look at the contents of the template for /etc/crontab:

[gordonr@smebuild crontab]$ pwd
/etc/e-smith/templates/etc/crontab

[gordonr@smebuild crontab]$ ls
00setup  10runparts  20statusreport  65_logrotate  email

Each of the files in that directory is a template fragment. The SME Server builds the /etc/crontab file by assembling those fragments and running them through the template processor.

To make your customization, create your own additional fragment by creating a file in this directory called 25templatedemo with the following contents:

# Template demo crontab entry:
*/20  *  *  *  *   root   /usr/bin/logger -t "Demo" "......"

Next time the SME Server regenerates the /etc/crontab file, it will contain your additional fragment. Starting the name with the prefix "25" forces the template fragment to go between the "20statusreport" and "65_logrotate" fragments. Force the /etc/crontab file to be generated immediately by typing the command:

expand-template /etc/crontab

If you look at the /etc/crontab file now, you should see your new fragment at the appropriate place, and your customization will take effect immediately (as cron notices when its configuration file has been changed). Check /var/log/messages to see the results.

To package this customization, you will need to create an RPM package that contains this single file, and ensures that the /etc/crontab template is expanded in the relevant events. You should also call one of these events in the RPM post-install section to ensure that the template is expanded without further action. Installing that RPM on any SME Server will cause the customization to occur, and will start printing the line of dots to the /var/log/messages logfile every 20 minutes.

The final point to note here is that if you remove your new file 25templatedemo and re-expand the /etc/crontab template, the crontab will go back to the way it was, and your customization will disappear cleanly. Therefore you should put a post-uninstallation script into your RPM package that runs the appropriate events to expand the templates once more. That will result in a package that installs and uninstalls cleanly.

Remember that for testing you can call expand-template directly in the post-install and post-uninstall sections, but released software should use the templates2expand mechanism to request template expansion in the relevant events.

Exercise 2: The magic of templates

For the next exercise, let us build on the first one. You have already created an exciting (o.k. not that exciting) new capability - the ability of the server to write dots into the log file. Let us now take advantage of the fact that the template processor can fill in values from the configuration database.

Edit the /etc/e-smith/templates/etc/crontab/25templatedemo file again, this time with the following contents:

# Template demo crontab entry:
*/20 * * * * root /usr/bin/logger -t "Demo3" "... { 
  use esmith::AccountsDB;

  $adb = esmith::AccountsDB->open_ro;
  $adb->get_prop('admin', 'ForwardAddress') || 'admin';
}

Once again, regenerate the template by typing:

expand-template /etc/crontab

If you look at the new /etc/crontab file, you will see that the template processor has replaced the block between the braces with the actual email address of the administrator, which is defined in the accounts database.

You could check that value with the db accounts show admin command. With this change to the template, the message which will go to the /var/log/messages log file every 20 minutes will contain the actual current administrative email address, rather than a hardcoded text message.

Now here is the exciting part. Click on the E-mail function in the server manager user interface, and change the administrative email address. If you look at /etc/crontab, you will see that it has been updated with the new email address! Every 20 minutes you will receive a new entry in the messages log which automatically reflects the new setting of the administrator email address.

So by simply creating a single file 25templatedemo, you have created quite a sophisticated customization that changes behaviour based on the system settings. And you have done so without affecting the rest of the system or requiring additional changes to the user interface.

The reason this works is subtle, but follows from the overall architecture. It is critical to study this example until you understand it thoroughly. If you understand exactly how this example works, you understand a substantial part of the SME Server architecture.

To understand this example in detail, let's start from the top by studying the user interface. The implementation section of the E-mail function can be found in the file /usr/lib/perl5/site_perl/esmith/FormMagick/Panel/emailsettings.pm. If you study the part of this script that gets executed when the user clicks Save, you will find some Perl code that saves the administrator e-mail address into the configuration database and signals the email-update event - thus informing the system that the email settings have changed:

sub change_settings_delivery
{
    my ($fm) = @_;
    my $q = $fm->{'cgi'};

    [...]

    my $admin_email = $q->param('AdminEmail');
    my $admin = $accounts->get('admin');
    if ($admin_email)
    {
        $admin->merge_props(
                EmailForward => 'forward',
                ForwardAddress => $admin_email,
                );
    }
    else
    {
        $admin->merge_props(
                EmailForward => 'local',
                ForwardAddress => '',
                );
    }

    [...]

    unless ( system( "/sbin/e-smith/signal-event", "email-update" ) == 0 )
    {
        $fm->error('ERROR_UPDATING');
        return undef;
    }
    $fm->success('SUCCESS');
}

When the email-update event is signalled, the SME Server executes all of the action scripts in the /etc/e-smith/events/email-update/ directory. The event also expands all templates as noted in the templates2expand/ hierarchy, including the /etc/crontab template.

That may all seem rather complicated, but it boils down to this: changing the administrator email address automatically rebuilds the /etc/crontab template - and if you have customized that template, your customization will automatically pick up the current administrator email address.

Note that if you wanted additional programs to execute whenever the email settings were changed, you would put all of those programs in the /etc/e-smith/events/actions/ directory, then create symbolic links to them from the /etc/e-smith/events/email-update/ directory. We use symbolic links because you may want your program triggered by other events as well as email-update, and so you create links from all of the relevant event directories back to your action program.

This system is, by design, extensible. For example, you could use this exact same type of customization to send an email message every hour containing the current IP address. This is left as an exercise to the reader.

Exercise 3: Using events and actions

In the SME Server, events are like callbacks in a programming language. The system signals an event whenever something interesting happens (e.g. a user is added, the IP address changes, etc.), which automatically executes all programs in the event directory. Therefore, any applications which need to know when a certain event is happening simply create a symbolic link from the event directory to a handler program, which will get executed whenever the event occurs.

In the previous exercise, we relied on the predefined events and actions in the SME Server to keep the /etc/crontab file up to date. In this example, we will create a new action script that will track the user accounts in the system. This script can be used as a template for any type of application that has its own notion of user accounts and needs to be driven by the existing user interface for adding, deleting, or modifying users.

Start by creating a new file called /etc/e-smith/events/actions/demo-user-tracking with the following contents:

#!/usr/bin/perl -w

# Set up Perl environment and libraries

package esmith;
use strict;
use warnings;
use esmith::ConfigDB;
use esmith::AccountsDB;

# Prepare to access configuration databases
my $db = esmith::ConfigDB->open_ro
    or die "Couldn't open ConfigDB\n";

my $accounts = esmith::AccountsDB->open_ro
    or die "Couldn't open AccountsDB\n";

# Read domain name from configuration database
my $domain = $db->get_value('DomainName');

# Read command line arguments
my $event = $ARGV [0];
my $id    = $ARGV [1];

# If no command line arguments, assume this is the initial setup
 # of all users. Process all accounts of type "user" (ignore groups,
# information bays, printers, etc.)

unless ($event and $id)
{
    for my $user ($accounts->users)
    {
        my $key     = $user->key;
        my $first   = $user->prop('FirstName');
        my $last    = $user->prop('LastName');

        system ("/usr/bin/logger", "-t", "Demo3",
                "Initializing user $key ($first $last) in domain $domain");
    }

    exit 0;
}

# If command line arguments are present, then this is a create, modify,
# or delete event, signalled by the SME Server event/action system.

my $user  = $accounts->get($id)
    or die "User $id does not exist\n";

my $first = $user->prop('FirstName');
my $last  = $user->prop('LastName');

if ($event eq 'user-create')
{
    system ("/usr/bin/logger", "-t", "Demo3",
            "Creating user $id ($first $last) in domain $domain");
}
elsif ($event eq 'user-modify')
{
    system ("/usr/bin/logger", "-t", "Demo3",
            "Changing user $id to ($first $last) in domain $domain");
}
elsif ($event eq 'user-delete')
{
    system ("/usr/bin/logger", "-t", "Demo3",
            "Deleting user $id in domain $domain");
}
else
{
    system ("/usr/bin/logger", "-t", "Demo3", "Ignoring $event event");
}

exit 0;

Make sure the permissions are correct:

chmod +x /etc/e-smith/events/actions/demo-user-tracking

Now create symbolic links so that this program is executed whenever a user is created, modified, or deleted. Make three symbolic links; one for each event directory:

cd /etc/e-smith/events
ln -s ../actions/demo-user-tracking user-create/S90demo-user-tracking
ln -s ../actions/demo-user-tracking user-modify/S90demo-user-tracking
ln -s ../actions/demo-user-tracking user-delete/S90demo-user-tracking

The S90 prefix ensures that the program will be executed after the standard actions (which typically have prefixes ranging from S15 to S80).

Study this program carefully. It uses many different SME Server capabilities. If invoked from the command line with no arguments, it will read all user accounts from the user database, fetch all the data fields associated with each user, and print this information to the log file. If invoked as an event, the SME Server will automatically pass it the event name and user id as command line arguments; in this case the program will print messages to the log file explaining that it is adding, modifying, or dropping the user.

Trying watching the log file by running the command:

tail -F /var/log/messages

or with the "View log files" panel in the server-manager. Use the standard SME Server user interface to add, modify, or remove users. You should see a stream of comments in the log file.

If you were creating an application that had its own database of user information, you would replace the logfile-writing code with your own code that initialized new users, modified them, etc. Then you would arrange for the RPM post-install script to execute:

/etc/e-smith/events/actions/demo-user-tracking

with no command line arguments. When the application is initially installed, it will immediately read all users from the database and set them up in your application. Then, as users are added, modified, or dropped over time - your code will be invoked each time, to update the application's private user database.

Exercise 4: Adding new configuration database parameters

New system configuration parameters can be spontaneously invented and added to the configuration database at any time. For example, let us return to our earlier exercise and parameterize the time interval for the log messages by introducing a new parameter called LogInterval.

You can write that parameter into the configuration database, as though it had always existed. For example, type this on the command line:

config set LogInterval 30

You can use config show LogInterval to show that it was set as intended. You can now edit the 25templatedemo file and replace the hardcoded number 20 with the LogInterval parameter. The resulting file will read:

# Template demo crontab entry:
*/{ $LogInterval } * * * * root /usr/bin/logger -t "Demo4" "... { 
  use esmith::AccountsDB;

  $adb = esmith::AccountsDB->open_ro;
  $adb->get_prop('admin', 'ForwardAddress');

} ..."

Now you can change the logger interval at any time by typing the following (replace the number 20 with whatever logger interval you want):

config set LogInterval 20
expand-template /etc/crontab

This ability to spontaneously introduce new configuration parameters is very important in the SME Server architecture. The configuration database is a high-level specification of how the overall system is supposed to behave for the end user. Configuration settings are like knobs on a stereo system. The templates, events, and actions, are the underlying machinery to carry out the user's wishes. When adding a new application to the system, it is important to be able to add new knobs on demand.

Now let us say that you want to introduce a parameter to enable or disable this logging function. At this point, you might start thinking of this logging activity as a service that you should be able to enable or disable. In this case the convention is to create a single service entry in the configuration database to manage both parameters.

To implement this, first delete the LogInterval parameter, which will no longer be needed:

db configuration delete LogInterval

Now create a service entry:

db configuration set loggerdemo service status enabled Interval 20

If you examine the configuration database you will see the new entry looks like this:

[root@gsxdev1 ~]# config show loggerdemo
loggerdemo=service
    Interval=20
    status=enabled

Now edit the 25templatedemo file to look like this:

# Template demo crontab entry
{
    my $status = $loggerdemo{status} || "disabled";

    return "# loggerdemo service is disabled."
        unless ($status eq "enabled");

    use esmith::AccountsDB;

    $adb = esmith::AccountsDB->open_ro;
    my $admin_email = $adb->get_prop('admin', 'ForwardAddress')
                        || 'admin';

    my $interval = $loggerdemo{Interval} || 10;

    $OUT = "*/$interval * * * * root /usr/bin/logger";
    $OUT .= " -t \"Demo4\" \"... $admin_email ...\"\n";
}

This is more complicated than the original template, but it is also more flexible. Note that the initial comment (# Template demo crontab entry) is hardcoded, but the line that follows is generated from the configuration database parameters. The code in the template retrieves the loggerdemo service entry, retrieves the required properties, and returns the appropriate output. To experiment, try different combinations of parameters:

db configuration setprop loggerdemo Interval 10
expand-template /etc/crontab

and

config setprop loggerdemo status disabled
expand-template /etc/crontab

and so on.

Exercise 5: Adding a user interface screen

Let us add a nice user interface screen to adjust the logger interval. Create a new file called /etc/e-smith/web/functions/loggerdemo, with the following contents:

#!/usr/bin/perl -wT
# vim: ft=xml:

#----------------------------------------------------------------------
# heading     : Demo
# description : Logger
# navigation  : 1000 1000
#----------------------------------------------------------------------

use strict;
use warnings;

use esmith::FormMagick::Panel::loggerdemo;

my $f = esmith::FormMagick::Panel::loggerdemo->new();
$f->display();

__DATA__
<form
    title="FORM_TITLE"
    header="/etc/e-smith/web/common/head.tmpl"
    footer="/etc/e-smith/web/common/foot.tmpl">

    <page name="First" pre-event="print_status_message()"
        post-event="change_settings">

        <field
            type="select"
            id="loggerdemo_Interval"
            options="10,20,30,40,50"
            value="get_interval()">
            <label>LABEL_LOGGERDEMO_INTERVAL</label>
        </field>

        <field
            type="select"
            id="loggerdemo_status"
            options="'disabled' => 'DISABLED', 'enabled' => 'ENABLED'"
            value="get_status()">
            <label>LABEL_LOGGERDEMO_STATUS</label>
        </field>


        <subroutine src="print_button('SAVE')" />
    </page>
</form>

The file above describes the panel layout, which is written in FormMagick XML. Further details about FormMagick can be fiound in the Section called An overview of FormMagick in Chapter 10.

Another file provides the implementation, which goes into /usr/lib/perl5/site_perl/esmith/FormMagick/Panel/loggerdemo.pm:

#!/usr/bin/perl -w 
package esmith::FormMagick::Panel::loggerdemo;

use strict;
use warnings;
use esmith::ConfigDB;
use esmith::FormMagick;

our @ISA = qw(esmith::FormMagick Exporter);

our @EXPORT = qw();

our $VERSION = sprintf '%d.%03d', q$Revision: 1.1 $ =~ /: (\d+).(\d+)/;

our $db = esmith::ConfigDB->open or die "Couldn't open ConfigDB\n";

sub get_status
{
    return $db->get_prop("loggerdemo", "status");
}

sub get_interval
{
    return $db->get_prop("loggerdemo", "Interval");
}

sub change_settings
{
    my $fm = shift;
    my $q = $fm->{'cgi'};

    $db->set_prop('loggerdemo', 'status', $q->param("loggerdemo_status"));
    $db->set_prop('loggerdemo', 'Interval', $q->param("loggerdemo_Interval"));

    unless ( system ("/sbin/e-smith/expand-template", "/etc/crontab") == 0 )
    {
        $fm->error('ERROR_UPDATING');
        return undef;
    }

    $fm->success('SUCCESS');
}

1;

Similarly to events and actions, the /etc/e-smith/web/functions/ directory is a repository of potentially available functions. To make the new function actually show up in the user interface, create a symbolic link to it from the web manager cgi-bin directory, as follows:

    cd /etc/e-smith/web/panels/manager/cgi-bin
    ln -s ../../../functions/loggerdemo loggerdemo

Now, make sure the permissions and ownership are correct so that the web server can run your new function:

cd /etc/e-smith/web/functions
chown root:admin loggerdemo
chmod 4750 loggerdemo

We also need to run a program to rebuild the navigation bar. This is only required when adding or removing functions from the manager, and is normally handled automatically. Let's do it manually for now:

/etc/e-smith/events/actions/navigation-conf

Try the server manager now, reloading the navigation bar in your browser, if necessary. You will see a new category called Demo and a new function within it called Logger. It will look a bit bare with some upper-case words which signify phrases which have not been localised. We'll fix that in a moment.

Try experimenting with it - it is a nice little user interface for playing with the logger customization. Every time you change the settings, notice that the /etc/crontab file is updated appropriately.

<lexicon lang="en-us">
    <!-- vim: ft=xml: 
    -->
    <entry>
        <base>FORM_TITLE</base>
        <trans>Logger demo</trans>
    </entry>

    <entry>
            <base>Demo</base>
            <trans>Demo</trans>
    </entry>

    <entry>
            <base>Logger</base>
            <trans>Logger</trans>
    </entry>

    <entry>
            <base>LABEL_LOGGERDEMO_STATUS</base>
            <trans>Status</trans>
    </entry>

    <entry>
            <base>ENABLED</base>
            <trans>Enabled</trans>
    </entry>

    <entry>
            <base>DISABLED</base>
            <trans>Disabled</trans>
    </entry>

    <entry>
            <base>SAVE</base>
            <trans>Save</trans>
    </entry>

</lexicon>

Exercise 6: Adding a new event type

Let us continue building on this example. Let us say that you want to add a hook to the logger demo, enabling other third party applications to receive a notification whenever the logger settings are changed. We need a new event type for this. Let us create a new event called loggerdemo-update:

mkdir -p /etc/e-smith/events/loggerdemo-update

The idea is that we will arrange for our user interface function to signal this new event whenever the settings are changed, instead of directly expanding the /etc/crontab file. Edit the /usr/lib/perl5/site_perl/esmith/FormMagick/Panels/loggerdemo.pm file and replace the line:

unless (system ("/sbin/e-smith/expand-template", "/etc/crontab") == 0)

with the line:

unless (system ("/sbin/e-smith/signal-event", "loggerdemo-update") == 0)

Now the loggerdemo user interface signals the new event whenever it saves the new settings to the configuration database. Next, we have to make sure that the event does what we need it to do - rebuild the /etc/crontab file.

cd /etc/e-smith/events/loggerdemo-update
mkdir -p templates2expand/etc
touch templates2expand/etc/crontab

Now the example should work just as before. You can edit the loggerdemo settings in the web manager, and see that the /etc/crontab file changes. But now other applications can also receive notifications of the loggerdemo-update event, by creating symbolic links from the /etc/e-smith/events/loggerdemo-update directory.

Exercise 7: Thought experiment - adding a new server application

You have now learned most of the machinery required for integrating a new server application into the SME Server. Consider a hypothetical chat server, with a configuration file called /etc/chatserv.conf.

You would first decide if the chat server needs any new settings. If so, you can create a new service entry in the configuration database to hold those settings.

You would then create a templated version of /etc/chatserv.conf, by creating the directory /etc/e-smith/templates/etc/chatserv.conf with at least one template fragment. The template should generate the correct version of /etc/chatserv.conf based on the configuration database settings. Experiment with different combinations of settings and manually running expand-template /etc/chatserv.conf.

Then create a web manager function (if one is required) enabling users to edit the settings. An event, for example chatserv-update should be signalled whenever the settings are changed. You would normally link into an existing event, but this is a thought experiment.

Then create the file /etc/e-smith/events/chatserv-update/templates2expand/etc/chatserv.conf to ensure that the configuration file is updated. You may also need to restart the chatserv program after changing its configuration, so you'll need a /etc/e-smith/events/chatserver-update/services2adjust/chatserv link which contains the word restart.

Finally, figure out when you'll need these scripts to be run. At a minimum, you'll want to run them whenever the event is signalled indicating that the user has saved new settings from the web manager. A symbolic link should be created, so that the new event triggers your new action scripts. Let's say you also need to reconfigure and restart your server each time a user is added. In that case, you would also create symlinks from the user-create event to your action scripts.

We are done - the new server has been integrated into the system. Note that every single one of these steps involved creating new files and directories, and reading/writing from the configuration database. No existing files were edited! You can easily imagine packaging these new files and directories into an RPM.

Customization guidelines

When creating applications:

• You can create new configuration parameters.

• You can add new configuration templates under /etc/e-smith/templates/.

• You can add fragments to any of the existing templates under /etc/e-smith/templates/.

• You can arrange for actions to be triggered upon package pre-install, post-install, pre-uninstall, or post-uninstall.

• You can link new actions into the events listed in the Section called Standard events and their arguments in Chapter 7.

• You can create new events to allow your feature set to be expanded. Do not create events for any other reason. They are not a replacement for function calls.

• You can arrange for new server programs to be started up at boot time.

• Typically you would expand a template at application post-install time, application post-uninstall time, and one or more of the other events.

• You can add new web functions to the navigation bar. Remember, panels should only retrieve and modify database values, perform validation, and signal events.

That is all! Applications should not make any extensions to the system other than these. For example, an application should not:

• Change the kernel or add new kernel modules.

• Edit configuration files directly - templates must be used.

• Link actions into any events other than the ones listed above or new events that you create. All built-in events other than the ones listed above are subject to change without notice in new SME Server releases.

• Directly access the per-user email store (i.e. the Maildir and related subdirectories within each user's home directory). This access should be performed via the IMAP server as the location and format of a user's home directory may change between releases.

• Take over the function of existing servers (i.e. shut down qmail and Apache and take over ports 25 and 80). The SME Server has features for proxying email and web requests to other servers on the system.

Do not expand templates at boot time. The only thing that should be happening during a normal system startup is to start servers. Templates should be expanded when it is necessary to change the system configuration (i.e. when a setting is changed, when the IP address changes, etc.) A normal shutdown or reboot should not trigger configuration changes. The bootstrap-console-save event will be run after a system reconfiguration, but will not run if the system does not require reconfiguration.

A quick introduction to RPMs

All SME Server software packages are distributed as RPM packages. This is the format used by CentOS and other major Linux distributions for distributing applications and other collections of files. The RPM system provides the ability to install, upgrade, remove and (importantly) verify the contents of installed packages.

An RPM essentially consists of an archive of all the files required by a piece of software. Additionally, it includes meta-information describing the software, and scripts which must be run to install or uninstall the software.

Meta-information stored in an RPM includes:

1. summary and description of the software

2. package name

3. version number

4. copyright information

5. category/group to which the software belongs

6. name and email address of the packager

7. pre-requisites to installing this package

8. ... and more

Selecting and creating RPMs for your application

Your application will typically depend on several components:

1. Software packages that are shipped as a standard part of the SME Server. You do not need to include any of these packages; they are always present in the runtime environment.

2. Software packages that are not a standard part of the SME Server, but that are required by your software, and would also be of general use in the runtime environment. For example: a Java runtime environment, libraries that enable communication with devices, etc. If possible, these packages should be made into separate packages, rather than being included in your application. This makes it easier to share them with other applications and enforces version compatiblity.

3. Software packages that are not a standard part of the SME Server, and that are specific to your software application (i.e. not generally useful in the runtime environment). This is the raw Linux version of your application without any specific SME Server integration code. For example, if your application is already available for Linux in the form of RPMs - these RPMs are what we are referring to. These are referred to as the application RPMs.

4. Any new files that you have created specifically in order to integrate your application into the SME Server runtime environment - should be packaged into a single RPM, as explained in the next section. This is referred to as the integration RPM.

So, if your application is based on Linux software that has already been packaged into RPMs, then you will need to create one new RPM:the integration RPM.

If, on the other hand, your application is based on Linux software that has not yet been packaged into RPMs, then you will probably need to create at least two RPMs: one or more application RPMs, and the integration RPM.

Finally, for simple customizations (such as the loggerdemo example earlier in this manual) there may be no application RPM at all. This would be typical if the point of the application is to change the server configuration without really adding a new software package. In this case you need only the integration RPM which contains the new template fragments, user interface screens, etc..

All files on the system, except for user data, must be installed by RPMs.

Setting up your RPM development environment

If you haven't done so already, set up an RPM development environment. If you are using an SME Server as your development environment, you will need to alter your user account to enable regular login. If you want to enable account "joe", then you would type the following commands from the root account:

chsh -s /bin/bash joe

db accounts setprop joe Shell /bin/bash

Then you should be able to log in to the server as user "joe", and get a Linux command line prompt. Log in, then type the following commands to set up your RPM work area:

cd ~/
mkdir -p rpms/{SRPMS,BUILD,SOURCES,SPECS,RPMS,lib}
mkdir -p rpms/RPMS/{i386,noarch}
echo "%_topdir $HOME/rpms" > ~/.rpmmacros

You will now find that you have a directory called rpms in which you will do your work. Under this are the following subdirectories:

As you prepare software to turn into RPMs, you will place files in these directories as appropriate. The following sections will describe what goes where as each item is covered.

Building an RPM

This section describes the process for building an RPM - step by step.

1. Choose a name and version number for your package. We are going to package the complete loggerdemo example and will use loggerdemo and 1.0.0 as the name and version number.

2. Collect all of the files which have been created in the previous sections into the /tmp/ directory. There is one additional file createlinks which looks like this:

   #!/usr/bin/perl -w
   
   use esmith::Build::CreateLinks qw(:all);
   use File::Basename;
   
   my $panel = "manager";
   
   panel_link("loggerdemo", $panel);
   
   

3. Create the directory hierarchy required for building the RPM. This is very close to the hierarchy on the installed system.

   # Change to the SOURCES directory
   cd ~/rpms/SOURCES
   
   # Remove old files (check that you don't need anything here!)
   rm -rf loggerdemo-1.0.0
   
   # Create new directory
   mkdir loggerdemo-1.0.0
   cd loggerdemo-1.0.0
   
   # The crontab template fragment
   mkdir -p root/etc/e-smith/templates/etc/crontab
   cp /tmp/25templatedemo root/etc/e-smith/templates/etc/crontab
   
   # The web panel description
   mkdir -p root/etc/e-smith/web/functions
   cp -p /tmp/loggerdemo !$
   
   # The web panel implementation
   mkdir -p root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel
   cp -p /tmp/loggerdemo.pm !$
   
   # The web panel English localisation
   mkdir -p root/etc/e-smith/locale/en-us/etc/e-smith/web/functions
   cp -p /tmp/loggerdemo-en !$
   
   # The createlinks auxiliary file
   cp -p /tmp/createlinks .

   Your directory structure should now look like this:

   [gordonr@sevendev1 loggerdemo-1.0.0]$ find . -type f
   ./root/etc/e-smith/templates/etc/crontab/25templatedemo
   ./root/etc/e-smith/locale/en-us/etc/e-smith/web/functions/loggerdemo
   ./root/etc/e-smith/web/functions/loggerdemo
   ./root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/loggerdemo.pm
   ./createlinks

4. Package the directory into a tarball:

   cd ~/rpms/SOURCES
   tar zcvf loggerdemo-1.0.0.tar.gz loggerdemo-1.0.0

5. Create the RPM specification "SPEC" file ~/rpms/SPECS/loggerdemo.spec which looks like this:

   %define name loggerdemo
   %define version 1.0.0
   %define release 01
   
   Summary: SME Server logger demo
   Name: %{name}
   Version: %{version}
   Release: %{release}
   License: GPL
   Group: Networking/Daemons
   Source: %{name}-%{version}.tar.gz
   Packager: Fred Frog <red@example.com>
   BuildRoot: /var/tmp/%{name}-%{version}-%{release}-buildroot
   BuildArchitectures: noarch
   
   %description
   Logger Demo sample application.
   
   %changelog
   * Thu Feb 2 2006 Fred Frog <fred@example.com>
   - 1.0.0-01
   - Original version
   
   %prep
   %setup
   
   %build
   perl createlinks
   
   DEFAULTS=root/etc/e-smith/db/configuration/defaults/loggerdemo
   mkdir -p $DEFAULTS
   
   echo "service"  > $DEFAULTS/type
   echo "enabled"  > $DEFAULTS/status
   echo "10"       > $DEFAULTS/Interval
   
   
   %install
   rm -rf $RPM_BUILD_ROOT
   (cd root ; find . -depth -print | cpio -dump $RPM_BUILD_ROOT)
   rm -f %{name}-%{version}-filelist
   /sbin/e-smith/genfilelist $RPM_BUILD_ROOT > %{name}-%{version}-filelist
   
   %clean
   rm -rf $RPM_BUILD_ROOT
   
   %post
   /etc/e-smith/events/actions/initialize-default-databases
   /etc/e-smith/events/actions/navigation-conf
   /sbin/e-smith/expand-template /etc/crontab
   true
   
   %postun
   /sbin/e-smith/expand-template /etc/crontab
   true
   
   %files -f %{name}-%{version}-filelist
   %defattr(-,root,root)
   
   

   Note the %post (post-installation) and %postun (post-uninstallation) statements which expand the /etc/crontab template after installing or uninstalling the RPM.

6. Check that your RPM will build OK with "build prepare":

   cd ~/rpms/SPECS
   rpmbuild -bp loggerdemo.spec

   The last line of output should be + exit 0 if rpmbuild is successful.

7. Run the rpmbuild command again to actually create your RPM with the "build all" options:

   rpmbuild -ba loggerdemo.spec

   If everything was successful, the last line of output should again be + exit 0.

8. The RPMs should have been generated and put into ~/rpms/RPMS/noarch/ as this program can run equally well on any platform. A source RPM should also exist in ~/rpms/SRPMS/.

9. Test your RPM by installing it on an SME Server test box.

   Note: RPMs need to be installed as root, but you should not log in as the root user. Instead, you should create a normal user and provide them with 'root' privileges via the sudo command. To provide full sudo rights to the user joe, use the su - -c /usr/sbin/visudo and add the following line to the end of the file:

   joe    ALL=(ALL) ALL

   You can then use the sudo to run commands as root when required. You will be prompted for your password (not the root password) which ensures that someone else isn't using your terminal.

   [joe@sevendev1 SPECS]$ sudo rpm -Uvh \
       ~/rpms/RPMS/noarch/loggerdemo-1.0.0-01.noarch.rpm
   Preparing...                ########################################### [100%]
      1:loggerdemo             ########################################### [100%]
   Migrating existing database mailpatterns
   Migrating existing database hosts
   Migrating existing database configuration
   Migrating existing database yum_repositories
   Migrating existing database networks
   Migrating existing database yum_updates
   Migrating existing database yum_installed
   Migrating existing database spamassassin
   Migrating existing database accounts
   Migrating existing database backups
   Migrating existing database yum_available
   Migrating existing database domains

   Note: In almost all instances you should use the -U (upgrade) option to rpm instead of the -i (install) option. The -i option should be reserved for special operations, such as installing kernels.

   The customization should be fully installed, and the /etc/crontab file should show the customization. Then remove the customization:

   # remove the RPM
   sudo rpm -e loggerdemo

   The customization should be completely gone, and the /etc/crontab file should look the way it did before.

#!/usr/bin/perl -w

use esmith::Build::CreateLinks qw(:all);

my $imap  = "/var/service/imap";
my $imaps = "/var/service/imaps";

templates2events("/etc/dovecot.conf", qw(bootstrap-console-save console-save));
templates2events("$imap/config",      qw(bootstrap-console-save email-update));
templates2events("$imaps/config",     qw(bootstrap-console-save email-update));

for my $event (qw(
    email-update
    ldap-update
    network-create
    network-delete
    ))
{
    safe_symlink("sigusr1", "root/etc/e-smith/events/$event/services2adjust/imap");
}

safe_symlink("daemontools", "root/etc/rc.d/init.d/imap");

for my $event (qw(post-upgrade))
{
    event_link("imap-relocate-maildirs", $event, "05");
}

service_link_enhanced("imap", "S55", "7");
service_link_enhanced("imap", "K45", "6");
service_link_enhanced("imap", "K45", "0");
service_link_enhanced("imap", "K45", "1");

Configuring your development environment

The SME Server source code is checked into CVS at SourceForge. The project has a SourceForge page at https://sourceforge.net/projects/smeserver/. All packages are built using mezzanine, and an overview of mezzanine is provided here: https://wiki.caosity.org/tiki-index.php?page=Package+Maintenance

export CVS_RSH=ssh # tell CVS to use ssh

# DO NOT set CVSROOT
sf_cvsroot=smeserver.cvs.sourceforge.net:/cvsroot/smeserver
sf_cvsroot_anon=:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver

alias mzgetsf="mzget --dir $sf_cvsroot"
alias mzimportsf="mzimport --dir $sf_cvsroot"

alias mzgetsf_anon="mzget --dir $sf_cvsroot_anon"

[gordonr@smebuild tmp]$ mzget smeserver-yum
cvs checkout: No CVSROOT specified!  Please use the `-d' option
cvs [checkout aborted]: or set the CVSROOT environment variable.
An unknown error must have occured, because the command returned 256

[gordonr@smebuild ~]$ mkdir ~/smeserver

[gordonr@smebuild ~]$ cd ~/smeserver

Modifying an SME Server package

[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys 
smeserver-yum-1.1.2-05

[gordonr@smebuild smeserver]$ mzgetsf_anon smeserver-yum
U smeserver-yum/ChangeLog
U smeserver-yum/F/smeserver-yum.spec
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch2
U smeserver-yum/P/smeserver-yum-1.2.0-ModifyUpdateDBs.patch
U smeserver-yum/S/smeserver-yum-1.2.0.tar.gz

[gordonr@smebuild smeserver]$ cd smeserver-yum/

[gordonr@smebuild smeserver-yum]$ ls
ChangeLog  CVS  F  P  S  smeserver-yum

[gordonr@smebuild smeserver-yum]$ mzclean
Cleaning and resyncing smeserver-yum, please wait....
Removing work....
Cleanup of smeserver-yum complete.

[gordonr@smebuild smeserver-yum]$ ls
ChangeLog  CVS  F  P  S

[gordonr@smebuild smeserver-yum]$ mzprep
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
You may now chdir to work to make changes.
Use "mzpatch -n <patch_name>" to generate a patch when done.

[gordonr@smebuild smeserver-yum]$ ls
ChangeLog  CVS  F  P  S  work

[gordonr@smebuild smeserver-yum]$ ls work/
smeserver-yum-1.2.0

[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/
createlinks      debugsources.list  smeserver-yum-1.1.2-26-filelist
debugfiles.list  root

[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/root/
etc  sbin  service  usr  var

[gordonr@smebuild tmp]$ diff -u yum-import-keys.orig yum-import-keys
--- yum-import-keys.orig        2006-05-22 16:28:51.734609534 +1000
+++ yum-import-keys     2006-05-22 16:28:39.006814950 +1000
@@ -30,6 +30,7 @@
 for my $key ( grep {!/^\./} readdir(DIR) )
 {
     warn "Importing key $key\n";
+    # TODO: Skip keys which have previously been imported

     system("rpm", "--import", $key) == 0 or
        warn "Couldn't rpm --import $key\n";

[gordonr@smebuild actions]$ cd ~/smeserver/smeserver-yum/

[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ModifyUpdateDBs.patch

[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs [server aborted]: "add" requires write access to the repository
You do not have write access to the master repository.
srctool:  Error:  Addition of P/smeserver-yum-1.2.0-ImportKeysComment.patch failed.
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ImportKeysComment.patch
smeserver-yum-1.2.0-ModifyUpdateDBs.patch

[gordonr@smebuild smeserver-yum]$ cat P/smeserver-yum-1.2.0-ImportKeysComment.patch
diff -Nur -x '*.orig' -x '*.rej' smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys
--- smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2005-09-26 12:30:54.000000000 +1000
+++ mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys       2006-05-22 16:35:57.752811633 +1000
@@ -30,6 +30,7 @@
 for my $key ( grep {!/^\./} readdir(DIR) )
 {
     warn "Importing key $key\n";
+    # TODO: Skip keys which have previously been imported

     system("rpm", "--import", $key) == 0 or
        warn "Couldn't rpm --import $key\n";

Summary: YUM, an rpm updater
%define name smeserver-yum
Name: %{name}
%define version 1.2.0
- %define release 05
+ %define release 05ff01

%changelog
+ * Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
+ - Initial work on avoiding duplicate RPM keys [SME: 1174]

* Mon May  1 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
- Remove stray yum.pm.orig file. [SME: 1350]

Patch0: smeserver-yum-1.2.0-DisplayStatus.patch
Patch1: smeserver-yum-1.2.0-ModifyUpdateDBs.patch
Patch2: smeserver-yum-1.2.0-DisplayStatus.patch2
+ Patch3: smeserver-yum-1.2.0-ImportKeysComment.patch

%prep
%setup
%patch0 -p1
%patch1 -p1
%patch2 -p1
rm root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/yum.pm.orig
+ %patch3 -p1

[gordonr@smebuild smeserver-yum]$ mzbuild

[...]
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.35037
+ umask 022
+ cd /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/build.mezz/BUILD
+ cd smeserver-yum-1.2.0
+ /bin/rm -rf /var/tmp/mezzanine-buildroot.18508
+ exit 0
Package build succeeded.  Output files are:

smeserver-yum-1.2.0-05ff01.src.rpm
smeserver-yum-1.2.0-05ff01.noarch.rpm

[gordonr@sevendev1 tmp]$ sudo rpm -Uvh smeserver-yum-1.2.0-05ff01.noarch.rpm
Preparing...                ########################################### [100%]
   1:smeserver-yum          ########################################### [100%]

SourceForge developer CVS access

If you are going to work on many bugs, or wish to help with the many tasks involved in a release, you should create a SourceForge developer account.

You do not need a SourceForge account in order to do development. You can contribute very effectively by producing patches and attaching them to the Bugzilla entries.

[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs add: use 'cvs commit' to add this file permanently
Patch added to SCM.  Use 'mzput' to upload to repository.

[gordonr@smebuild smeserver-yum]$ mzput -m 'See changelog'

Leveraging the provisioning system for users, groups, and i-bays

One of the themes in the SME Server is that concepts such as users, groups, and shared information (information bays) are simplified and reused in the user interface. SME Server users are email users, filesharing users, web users and users for any other sofware installed on the system.

For example, in the user interface you can create an information bay called salesdata representing information of interest to the sales team. Creating the information bay automatically reconfigures Samba and Netatalk to share salesdata as a new shared folder, reconfigures Apache to present http://www.example.com/salesdata/ as a new part of the web site, and reconfigures the FTP server - so that the information can be accessed by logging in as user salesdata.

Another example of this type of concept-reuse is that you can create a group called marketing that will, among other things, create an email alias called marketing to automatically forward email to all the group members. This group can also be used as a unit of information-sharing.

In order to enable this concept-reuse, there are certain namespace restrictions. You cannot have a user account and an information bay with the same name - since there would be ambiguity when logging into the FTP server. You cannot have a user account and a group with the same name either - since there would be ambiguity when sending email to the server.

To enforce these restrictions, the SME Server defines a concept of account. Users, groups, and information bays are all different types of account. No two accounts can have the same name. The account list is maintained in the accounts database.

Whenever a user, group, or information bay is created, the following steps are performed automatically by the SME Server:

1. Check if there is an existing account (of any type) with the same name. If so, display an error and terminate.

2. If there was no error, then create a new accounts database entry. The entry contains the name of the account, its type (e.g. user, group, ibay), and all associated properties.

3. Signal the create event for that account type - user-create, group-create, ibay-create, and so on.

4. The actions for that event will then do all the work to set up the account - creating underlying user accounts if necessary, creating groups and directories, reconfiguring services, and so on.

The SME Server supports the following account types:

When creating applications, you should always try to make use of the built-in SME Server account types. If your application has any concept of users, groups, or shared data - try to make your application use the built-in SME Server mechanisms for all of these.

Programmatically creating users, groups, and i-bays

You can create users, groups, and i-bays by creating database defaults, or through code. Refer to the useraccounts, groups and ibays panels for examples of how to create these items. You can also create accounts with simple shell scripts.

For example, here is a shell script to create a user (account "abc", name "Albert Collins"):

#!/bin/sh

PATH=/sbin/e-smith:$PATH

if db accounts get abc >/dev/null
then
    echo "abc already exists in the accounts database"
    db accounts show abc
    exit 1
fi

db accounts set abc user PasswordSet no
db accounts setprop abc FirstName Albert 
db accounts setprop abc LastName Collins
db accounts setprop abc EmailForward local

signal-event user-create abc

Note that we could have provided all of the properties to the set command and created the record in one step. Here's the same example using the Perl libraries

#!/usr/bin/perl -w

use strict;
use warnings;
use esmith::AccountsDB;

my $db = esmith::AccountsDB->open or die "Couldn't open AccountsDB\n";

my $abc = $db->get("abc");

if ($abc)
{
    die "abc already exists in the accounts database\n" .
        $abc->show . "\n";
}

$db->new_record("abc",
        {
            type         => 'user',
            PasswordSet  => 'no',
            FirstName    => 'Albert',
            LastName     => 'Collins',
            EmailForward => 'local',
        });

unless ( system("/sbin/e-smith/signal-event", "user-create", "abc") == 0 )
{
    die "user-create abc failed\n";
}

exit 0;

Reserving accounts to avoid conflicts with user, group, or i-bay names

If your application creates a new directory within your web site e.g. http://www.example.com/magicstuff/, you should make sure the name isn't also used for an information bay, since that would create a conflict. Simply reserve the name by creating a urlaccount. This can be done by creating a defaults file:

cd /etc/e-smith/db/accounts/defaults/

mkdir magicstuff
cd magicstuff

echo url >type

If you package the file in your RPM, the account will be created automatically. To test your change before packaging, you'll need to tell the SME Server to reconfigure the databases:

/etc/e-smith/events/actions/initialize-default-databases

db accounts show magicstuff

Adding new account properties

Just as you can spontaneously introduce new configuration settings you can spontaneously introduce new properties as well.

For example, let's say that your application needs a concept of cell phone number stored for each user account. This is not a standard property in the SME Server. Your application can simply choose a name for the new property, e.g. CellNumber, and immediately start reading and writing that property for the various users - as though the property had always existed.

If you read from a non-existant property, an empty string is returned for shell scripts and the undef value is returned when using the Perl interfaces. If you write to a non-existent property, it is spontaneously created in the accounts database.

Here is an example of a user interface screen which allows you to edit cell phone numbers for each user account. As before, the form descriptions goes in /etc/e-smith/web/functions/cellnumbers :

#!/usr/bin/perl -wT
# vim: ft=xml ts=4 sw=4 et:
#----------------------------------------------------------------------
# heading     : Collaboration
# description : Cell numbers (fm)
# navigation  : 3000 3150
#----------------------------------------------------------------------
use strict;
use esmith::TestUtils;
use esmith::FormMagick::Panel::cellnumbers;

my $fm = esmith::FormMagick::Panel::cellnumbers->new();

$fm->display();

__DATA__
<form title="FORM_TITLE" 
    header="/etc/e-smith/web/common/head.tmpl" 
    footer="/etc/e-smith/web/common/foot.tmpl">

    <page name="First" pre-event="print_status_message()">

        <description>FORM_DESCRIPTION</description>

        <subroutine src="print_cellnumbers_table()" />
    </page>

    <page name="CELLNUMBERS_PAGE_MODIFY"
            pre-event="turn_off_buttons()"
            post-event="modify_cellnumber()" >

        <description>MODIFY_TITLE</description>

        <field type="literal" id="User" >
            <label>LABEL_USER</label>
        </field>

        <field type="literal" id="FullName">
            <label>LABEL_FULLNAME</label>
        </field>

        <field type="text" id="CellNumber">
            <label>LABEL_CELLNUMBER</label>
        </field>

        <subroutine src="print_button('SAVE')" />
    </page>
</form>

And the form implementation goes in /usr/lib/perl5/site_perl/esmith/FormMagick/Panels/cellnumbers.pm :

#!/usr/bin/perl -w 
package    esmith::FormMagick::Panel::cellnumbers;

use strict;

use esmith::FormMagick;
use esmith::AccountsDB;
use esmith::ConfigDB;

use Exporter;
use Carp qw(verbose);

use HTML::Tabulate;

our @ISA = qw(esmith::FormMagick Exporter);

our @EXPORT = qw();

our $db = esmith::ConfigDB->open();
our $adb = esmith::AccountsDB->open();

sub new 
{
    shift;
    my $self = esmith::FormMagick->new();
    $self->{calling_package} = (caller)[0];
    bless $self;
    return $self;
}

sub print_cellnumbers_table
{
    my $self = shift;
    my $q = $self->{cgi};

    my $cellnumbers_table =
    {
       title => $self->localise('CURRENT_LIST_OF_CELLNUMBERS'),
       
       stripe => '#D4D0C8',

       fields => [ qw(User FullName CellNumber Modify) ],

       labels => 1,

       field_attr => {
                       User => { label => $self->localise('USER_LABEL') },

                       FullName => { label => $self->localise('FULLNAME_LABEL') },

                       CellNumber => { label => $self->localise('CELLNUMBER_LABEL') },

                       Modify => { 
                                   label => $self->localise('MODIFY'),
                                   link => \&modify_link },
                                 }
           };

    my @data = ();

    my $modify = $self->localise('MODIFY');

    for my $user ($adb->users)
    {
        push @data, 
            { 
              User => $user->key,

              FullName => $user->prop('FirstName') . " " . 
                          $user->prop('LastName'),

              CellNumber => $user->prop('CellNumber') || '',

              Modify => $modify,
            }
    }

    my $t = HTML::Tabulate->new($cellnumbers_table);

    $t->render(\@data, $cellnumbers_table);
}

sub modify_link
{
    my ($data_item, $row, $field) = @_;

    return "cellnumbers?" . 
            join("&", 
                "page=0",
                "page_stack=",
                "Next=Next",
                "User="     . $row->{User},
                "FullName=" . $row->{FullName},
                "CellNumber=" . $row->{CellNumber},
                "wherenext=CELLNUMBERS_PAGE_MODIFY");
}

sub modify_cellnumber
{
    my $self = shift;
    my $q = $self->{cgi};

    my $user = $adb->get( $q->param('User') );

    $user->set_prop('CellNumber', $q->param('CellNumber'));

    return $self->success('SUCCESSFULLY_MODIFIED');
}

1;

Save the two files in the correct locations and then set the correct permissions and ownership:

cd /etc/e-smith/web/functions
chown root:admin cellnumbers
chmod 4750 cellnumbers

Then create a symbolic link to the script from the web manager cgi-bin/ directory:

cd /etc/e-smith/web/panels/manager/cgi-bin
ln -s ../../../functions/cellnumbers cellnumbers

/etc/e-smith/events/actions/navigation-conf

If you refresh the navigation bar, you will see a Cell numbers screen, which can be used to edit cell phone numbers for each user.

You could easily package this into an RPM and would just need the cellnumbers description, the cellnumbers.pm implementation and the symbolic link in the RPM. If you installed this application on any SME Server you could immediately start entering cell phone numbers for each user.

Using the LDAP server

The SME Server automatically creates and maintains an LDAP address book. The LDAP server listens for requests on port 389, which is the standard TCP/IP port for LDAP. At this time, the LDAP server should be considered read-only as it is generated from the system configuration and accounts data. Changes to the LDAP schema will be backed up and restored, but major system reconfiguration may reset the LDAP database to the default schema.

Data backup

The SME Server supports two methods for data backup. For light-usage sites, end users can use their web browser to select a backup to desktop option; this creates a compressed file of the configuration databases and all user data on the server, and uploads it to the user's desktop via their web browser.

For heavier-usage sites, automatic nightly tape backup can be configured.

Third party application writers do not need to make special backup arrangements. All that is required is to ensure that all data files are placed within the standard directories that are backed up. All files and directories within the /home/e-smith/files/ tree are always backed up by all of the SME Server backup mechanisms.

There is a pre-backup event which is signalled before a backup is performed. This can be used to shutdown applications or databases to ensure that a consistent state is backed up. The SME Server automatically performs an ASCII export of all MySQL databases in pre-backup event.

There is a corresponding post-backup event which is signalled after the backup has been performed. This can be used to restart services after the backup.

Using the MySQL database

The SME Server provides a standard method for performing MySQL database initialization and migration. This is done by creating files in the /etc/e-smith/sql/init/ directory. These files are run automatically when MySQL is started, and deleted if they run successfully.

A separate MySQL database and one or more database users should be created for each application. The database password should be stored in the configuration database and either retrieved from the configuration database by the application or passed to the application via an httpd.conf fragment. The password should be automatically generated, unique to this server and this application, and stored as a property in the configuration database for later use in database scripts.

The MySQL root is automatically generated and configured for command-line MySQL use by the root system user. The MySQL root password should only be used for database maintenance such as creating and deleting databases and performing database backups.

First choose a name for your database, as well as a username to access the data. For example, let's say your database is called loggerdemo, the username is loguser and the password is $loggerdemo{DbPassword}. A migrate fragment like this might be used to create the password:

{
    my $rec = $DB->get('loggerdemo')
        || $DB->new_record('loggerdemo', {type => 'service'});

    my $password = $rec->prop('DbPassword');
    return "" if $password;

    use MIME::Base64;

    my $rand     = sprintf("%08d", int(1_000_000_000 * rand()));
    my $password = MIME::Base64::encode($rand, "");

    $rec->set_prop('DbPassword', $password);
}

Then create a template which generates a file in the /etc/e-smith/sql/ directory, and put the relevant SQL commands in that file. The SQL commands should set up the application's username and retrieve the database password from the configuration database. It creates the new MySQL database and any tables required by your application. Write these SQL commands using the IF NOT EXISTS clause so that they do nothing if the tables have already been created. For example, you might create the template /etc/e-smith/templates/etc/e-smith/sql/loggerdemo-create-schema.sql with the following contents:

# Create the user account and password. (This is harmless if the
# user account and password already exist.) Note the reference
# to the 'loggerdemo' database which will be created in the next
# few statements.
  
USE mysql;
  
REPLACE INTO user (host, user, password)
    VALUES ('localhost', 'loguser', PASSWORD ('{ $loggerdemo{DbPassword} }'));
  
REPLACE INTO db (host, db, user,
                   select_priv, insert_priv, update_priv,
                   delete_priv, create_priv, drop_priv )
     VALUES ('localhost', 'loggerdemo', 'loguser',
             'Y', 'Y', 'Y', 'Y', 'Y', 'Y');
  
FLUSH PRIVILEGES;
  
# Create 'loggerdemo' database. (Do nothing if the database
# already exists.)
  
CREATE DATABASE IF NOT EXISTS loggerdemo;
  
# Create log_entry table within the 'loggerdemo' database.
# (Do nothing if the table already exists.)
  
USE loggerdemo;
  
CREATE TABLE IF NOT EXISTS log_entry
(
    entry_message  varchar(60),
    entry_time     datetime
);

Include the migrate fragment and your template in your RPM. Note that the password generated in this way is unique to this server and this application and automatically stored in the configuration database for later use. This means that it is backed up and restored through the normal server operations.

In the post-installation section of your RPM, expand the template, and run the /etc/rc.d/init.d/mysql.init script. For example the post-installation section of your RPM SPEC file might look like this:

%post
expand-template /etc/e-smith/sql/init/loggerdemo-create-schema.sql
/etc/rc.d/init.d/mysql.init
true

Installing this RPM will create the /etc/e-smith/sql/loggerdemo-create-schema.sql templates (because it is part of the RPM), and the post-installation actions will expand the template and run the mysql.init script, which will execute the schema creation commands and delete the generated file. When the RPM installation is finished, the database schema will have been created, and the MySQL database will be ready to process SQL commands from your application.

It is also possible to perform MySQL initialization in languages other than SQL, for example if the logic is better suited to another language, simply by creating a file in the /etc/e-smith/sql/init/ directory. The file must be executable and not have a .sql extension. For example, the template expansion might generate this file:

#! /bin/sh

exec mysql < /home/httpd/html/horde/scripts/db/mysql_create_tables.sql

You can use the templates.metadata mechanism to ensure the correct permissions on the generated file. Remember, the files are removed from the /etc/e-smith/sql/init/ directory if they run successfully.

It is important to think through what will happen when your application is installed, uninstalled, reinstalled, or upgraded. The instructions described above do not specify any uninstallation procedure - therefore the database tables will be left unchanged if your application is removed, reinstalled, or upgraded. If you want the data to be deleted when the application is removed, create a post-uninstallation script using the same technique as the post-installation script.

The instructions above apply to an application with a schema that does not evolve. If you create a new version of your application that requires schema changes, your post-installation script will have to migrate the database from the old to the new schema. In that case you have two options. Say the original version of the application is 1.0, and the new version is 1.1.

1. The first option is to release two versions of the 1.1 application - one for new installations (containing SQL commands to create a new schema), and a second version for upgrading 1.0 installations (containing SQL commands to upgrade the 1.0 schema). The RPM mechanism allows you to specify dependencies to ensure that only the correct version of each RPM can be installed on a given SME Server.

2. The better option is to change the template so that it includes the appropriate MySQL code to query the database and automatically determine whether to migrate an existing schema or create a new one. The e-smith-horde package contains a number of MySQL database initialisation and migration scripts which may be useful to study.

Sending email messages

If your application needs to send an email message, it should use the SMTP protocol and send the message through the local SMTP server (connect to localhost, port 25).

There are many toolkits available to make this simpler, for example the Mail::Send library (see perldoc Mail::Send) if you are sending the message from a Perl program.

Managing the firewall

The SME Server approach provides better security than a typical firewall, because the SME Server is managed automatically. Conventional firewalls have complex user interfaces, and require a system administrator to choose policies (e.g. which services should be permitted, which ports should be forwarded, etc.) The SME Server firewall has no user interface. It automatically generates the best ruleset that is consistent with the server settings, and is automatically regenerated whenever the server settings are changed.

config set myservice service TCPPort 4321 access public status enabled

signal-event remoteaccess-update

cp /etc/rc.d/init.d/masq /tmp

config setprop myservice access private

signal-event remoteaccess-update

diff -u /etc/rc.d/init.d/masq /tmp/masq

[root@gsxdev1 esmith]# diff -u /tmp/masq /etc/rc.d/init.d/masq
--- /tmp/masq   2006-02-02 13:14:09.000000000 +1100
+++ /etc/rc.d/init.d/masq       2006-02-02 13:14:13.000000000 +1100
@@ -340,9 +340,7 @@
     /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 389 \
        --destination $OUTERNET --jump denylog

-    # myservice: TCPPort 4321, AllowHosts: 0.0.0.0/0, DenyHosts:
-    /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 4321 \
-       --destination $OUTERNET --src 0.0.0.0/0 --jump ACCEPT
+    # myservice: TCPPort 4321, AllowHosts: , DenyHosts:
     /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 4321 \
        --destination $OUTERNET --jump denylog

config setprop myservice access public

config setprop myservice AllowHosts 1.2.3.4,100.100.100.0/24

signal-event remotaccess-update

diff -u /etc/rc.d/init.d/masq /tmp/masq

[root@gsxdev1 esmith]# diff -u /tmp/masq /etc/rc.d/init.d/masq
--- /tmp/masq   2006-02-02 13:14:09.000000000 +1100
+++ /etc/rc.d/init.d/masq       2006-02-02 13:22:32.000000000 +1100
@@ -340,9 +340,11 @@
     /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 389 \
        --destination $OUTERNET --jump denylog

-    # myservice: TCPPort 4321, AllowHosts: 0.0.0.0/0, DenyHosts:
+    # myservice: TCPPort 4321, AllowHosts: 1.2.3.4,100.100.100.0/24, DenyHosts:
     /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 4321 \
-       --destination $OUTERNET --src 0.0.0.0/0 --jump ACCEPT
+       --destination $OUTERNET --src 1.2.3.4 --jump ACCEPT
+    /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 4321 \
+       --destination $OUTERNET --src 100.100.100.0/24 --jump ACCEPT
     /sbin/iptables -A $NEW_InboundTCP --proto tcp --dport 4321 \
        --destination $OUTERNET --jump denylog

Starting up programs automatically upon system boot

If your package implements a server or daemon, you will probably want it to be started automatically when the system boots.

The SME Server boots in runlevel 7, so you can get an idea of the startup processes by listing the contents of /etc/rc.d/rc7.d.

These are similar to the init scripts you may be familiar with from other Linux systems, with one important differnce. Instead of pointing to scripts within /etc/rc.d/init.d, all of those init entries are links to /etc/rc.d/init.d/e-smith-service. This is a wrapper which checks the configuration database to see if the service is supposed to be running and if so, starts the service from /etc/rc.d/init.d/whatever.

So for example, you might have:

S90squid -> /etc/rc.d/init.d/e-smith-service

The e-smith-service script looks up the name it was invoked with (S90squid), drops the prefix (leaving squid), checks the configuration database for the "squid" service, then if it's supposed to run, does:

/etc/rc.d/init.d/squid start

Here is the step-by-step procedure for making your package start up a program called myserver at boot time.

1. First, create the traditional init script /etc/rc.d/init.d/myserver which can be run with the command-line arguments "start" or "stop" to perform the appropriate action on the server. Look at other init scripts in the same directory for examples. This script should be included in your RPM.

   Note: If your service is managed by runit, all you need is a link to the daemontools script.

2. Second, create a symbolic link as shown below, choosing the two-digit number after the letter S to control the startup order of the server programs. Include this symbolic link in your RPM.

   /etc/rc.d/rc7.d/S55myserver -> /etc/rc.d/init.d/e-smith-service

   These two steps are typical for any Linux/Unix server application, except that the symbolic link traditionally points directly to the init script, rather than to e-smith-service. Remember, we want to link to e-smith-service to ensure that a disabled service does not start at boot time.

3. Third, let's assume for now that myserver should be enabled by default, and so start at boot time. You just need to create some properties in the configuration database to make that happen:

   cd /etc/e-smith/db/configuration/defaults
   mkdir myserver
   cd myserver
   
   echo service >type
   echo enabled >status

   For testing, you will also need to run initialize-default-databases to load these new defaults.

4. Your RPM can also start the service in the %post section, but you need to be very careful to only do this in run-level 7. The same %post section is run during installation from CDROM, and we do not want services started during that installation. They will most likely fail and may cause the CD install to fail.

All of these steps result in the server starting automatically upon installation of the RPM, and whenever the server is rebooted.

Care should also be taken for the RPM to uninstall cleanly. The service should be stopped and marked not to be restarted and so your RPM should contain the following lines in the pre-uninstallation script:

%preun
/sbin/e-smith/db configuration setprop myserver status disabled
/etc/rc7.d/S55myserver stop
true

The /service/myserver symbolic link is owned by the RPM, and when it is removed, runsvdir will soon notice and kill the runsv supervision process. The final true command ensures that a failure from the other commands won't cause the removal of the RPM to fail. Note that these steps cannot be in the post-uninstallation script, since some of the required files may have been removed by that time.