CDDB Protcol Server v1.3.1 HOWTO

Copyright (c) 1996  Steve Scherf
A Moondog Software Production

Author: Steve Scherf
Email:  steve@moonsoft.com
Rev:	@(#)$Id: CDDBD_HOWTO,v 1.1 1996/12/13 23:35:22 steve Exp $


Introduction:
-------------

The CDDB daemon, cddbd, is an all-purpose Compact Disc database manager
for Unix designed to work with the popular "xmcd cddb" database. This document
describes how to configure, compile and run the daemon in its various
operating modes.


Table of Contents:
------------------

Section 1:   Compiling the Daemon
Section 2:   Installing the Daemon
Section 3:   Operating Modes
Section 3.1: Running the Daemon as an Internet CD Database Server
Section 3.2: Configuring the Daemon to Accept Automatic Updates
Section 3.3: Configuring the Daemon to Accept Email Submissions
Section 3.4: Configuring the Daemon to Transmit Entries to Remote Sites
Section 4:   How Clients Should Use the Server
Section 5:   Getting Help
Appendix A:  The Access File
Appendix B:  The Sites File


1. Compiling the Cddb Daemon
-----------------------------

Before compiling, you must first run the config.sh script. This is only
the preliminary configuration step, so don't worry about configuration
and installation details yet. The script will ask you for the path to
the directory in which the server access fill will be located. This path
is compiled into the server, but can be overridden when you actually
run the server. The path should generally be the path to the directory
which will you will be specifying later in the installation phase as the
daemon's work directory.

Run the script by typing, "sh config.sh" and answer the single question.
You are now ready to compile.

You must run this step before any other, or that step will fail. Once you have
configured cddbd, however, you can run the other steps as often as you like.

To compile cddbd, type, "make". There is a simple stock makefile included
in the distribution that should work on most Unix and Unix-like systems.
The program is fairly vanilla and should compile without additional flags
or libraries, but there is no guarantee of this. Quite often socket-related
libraries such as libnsl, libsocket or libbsd, among others, need to be
specified; this is because Internet socket calls are not automatically
included by many Unix compilers.

Add compiler flags to the CFLAGS makefile variable, and loader options to
the LDFLAGS variable as necessary to make the daemon compile.

If you are unsuccessful, there is also an Imakefile included in the
distribution. If you have an X development environment properly installed
on your system, you should be able to type "xmkmf ; make -f Makefile" to get
a successful build of cddbd. Although the daemon is not an X application,
this should still work.

The daemon was developed with gcc 2.6.2, and passes lint (for the most
part). It has also been compiled and tested with several other ANSI C
compilers on various Unix platforms, so you should have little trouble
getting to this point if you use a good ANSI compiler.


2. Installing the Daemon
------------------------

To install cddbd, type, "sh install.sh". The script will ask you questions
about where you want things to point, and which key features you want
enabled. This script will install the binary and create all needed directories
and files.

You might want to install the xmcd-cddb CD database itself before proceeding
with this step, or the install script will complain. The installation will
still work, but you need to be sure to install the database at the location
you specify during this step or the server will be unable to locate the
database. See the README file included with the cddbd distribution for
details on how to obtain the xmcd-cddb database.

The database tar distribution is created with files publically writable and
owned by user "bin". Be sure to change these to the proper user ID and modes
before running the daemon; this is usually your own user ID, but doesn't
necessarily have to be. You can configure the server to run with the database
owned by a user ID other than the owner of the daemon if you wish. Setting up
the server to run with the database owned by someone else is described below.

In order to properly answer the installation questions, you should have some
idea of how you will be using the daemon. There are several operating modes
that you may wish to configure the daemon for, and you should decide which
of these you will be using before you proceed with installation. However,
if you make a mistake during installation, you may reinstall the server at
any time. You can also edit the access file by hand to effectively configure
the server for the different operating modes. The various modes are described
in detail in the following sections, so you might want to read ahead a bit
before installing the daemon.

As you run the install script, it will ask various questions with default
answers in brackets. If you press return in response to questions, the
specified default will be used.

If you are not the super-user when installing the daemon, the script will
ensure that you know what you're doing. It is possible to install and run
the server as a non-root user, so don't fret if you're just a normal user
or simply don't want to run the server as root. Just tell the script to
continue when it asks.

The script will ask for various pathnames needed to run the daemon. Just
press return to accept the listed default. The "binary directory" is where
the daemon itself will be installed. The "work directory" is where the
daemon puts the files it needs to operate. The "database directory" is
where the you installed (or will install) the CD database.

When the script asks if you want to enable logging, you should answer yes
unless you've got a good reason not to. The server automatically limits the
size of the log file, so you needn't worry about the log file growing too
large if you don't watch it.

You will be asked if you will be operating a master database hub. You probably
want to answer no, unless you will be maintaining remote database mirror sites.
If you answer yes, you will be asked for an SMTP server host. This should
probably be "localhost", unless your local SMTP server is somewhere else. If
you don't have an SMTP server, you should just choose localhost. You won't
be able to run the daemon in mail filter mode, but you will still be able
to run as a master hub. When asked for an email address, you should specify
the address of the user who should receive submission rejection notices
in the event of submission errors. This address must be specific enough for
the daemon to be able to properly submit mail to the desired user from the
system that the daemon will run on.

The next set of questions gets a little more complicated. The script asks what
user/group will own the daemon, what the daemon work file permissions should
be, what user/group owns the database, and what the database file permissions
should be. In general, you should just accept the defaults. You have little
choice with respect to daemon ownership unless you have root capabilities,
as the install script will attempt to set file ownerships and permissions
according to the selections that you make. If you are not root and you specify
an owner other than yourself, the install script will probably be unable to
create files accordingly.

If you don't own the database, do not despair. It doesn't matter too much
what you choose for the database settings in this case. Unless you own
the database, you will not be able to modify it, and that is the only time
these values matter. In this case, you will only be able to use the daemon
for read-only operations. In effect, you will be limited to using the daemon
as a database server. Other operations, such as updates and check-and-modify,
require that you have the proper database permissions.

If you do own the database, or will be running the daemon as root, you
should specify the correct owner of the database or the daemon may change
ownership of database entries to the user you specify. It is suggested that
you use the default file and directory permissions (644 and 755 respectively)
for the database, as the database is distributed with permissions (666 and 777)
set to allow anyone to modify files. This is inherently unsecure and the
database will correct permissions as needed should you accept the default.

Note: you should instead use 666 and 777 if the database directory will be
shared with users of xmcd, as xmcd expects to be able to write to the
database when saving new or modified entries. It is not suggested that you
share the database, as xmcd may interfere with the operation of the daemon.
The daemon depends upon special file locking for certain operations, and
xmcd does not observe this locking. This will be especially important in
future versions of the daemon. Also, files added to the database by xmcd
will not be found in the fuzzy matching hash file unless you manually
regenerate it.

In all cases, if you wish you can specify a numerical user or group ID
instead of a name. This may be necessary on sites where NIS is running,
as there seems to be a bug in some versions of the getgrent library call
which causes it to return local group IDs instead of NIS IDs. It is also
useful if your system's passwd or group files are incorrect.

The next set of questions concern configuring the server for automatic
database updates. You probably want to answer no to this question unless
you are sure this is what you want. In order for this to work, you must
make arrangements a master site (currently only cddb.moonsoft.com) to transmit
entries to your site. If this is what you want, answer yes. Accept
the default database feed site, which just happens to be moonsoft.com, unless
you will be receiving updates from some other site. There's little point in
choosing a different site unless you're setting up your own distribution
scheme.

The script next asks if you wish to create a "message of the day" file.
The information in this file is displayed as the output of the "motd" server
command. If you answer yes, the next question will ask what editor you wish
to use and then launch that editor. Enter the desired information, save, and
quit from the editor.

The script will then install the important daemon files. Watch for errors,
as these files are important.

The last question asks whether you wish to create the fuzzy matching hash
file. You should answer yes to this, unless you won't be using the daemon
as a server. If you don't, the daemon will complain when run as a server,
although it will work for the most part. This step takes some time, as
considerable CPU and disk bandwidth are required. You can ignore most errors,
if any, as there may be some questionable database entries (depending on
what version of the database you are using). These are just ignored, and
simply won't appear in the hash file.

You are now done installing! There are a few other steps required to get
the daemon running, depending on what you will be using it for. Refer to
the "Operating Modes" section for information on what you still need to
do.

Please note:

- If the daemon is set up to run setuid/setgid, whether root or otherwise,
  DO NOT allow users to execute it directly. If you do, users can
  damage or destroy files on your system. While this may seem like common
  sense, it is an easy fact to overlook. In general, there is little
  reason to allow users to execute the daemon directly, regardless of
  how the daemon is configured. You should ensure that the daemon is only
  accessible to users through inetd and/or email. You have been warned.


3. Operating Modes
------------------

The server has several basic functions:

- Internet server:

  Functions as a server for Internet-aware CD-player programs that wish to
  perform database lookups.

- Database manager:

  Automatically maintains and verifies the CD database.

- Email processor:

  Accepts new database submissions via email and integrates them into the
  database.

The four sections below explain how to configure and run the server in the
various operating modes.


3.1 How to Run the Daemon as an Internet Server
-----------------------------------------------

The main function of the daemon is to act as an Internet CDDB server. In
this mode, the daemon allows users to connect via numerous popular CD player
programs and automatically download database entries. Generally, when a
user inserts a CD into his CD ROM drive, his CD player software will perform
a query of an Internet server whose address the user specifies. If the server
locates a match, the software downloads it from the server and displays it
for the user.

There are numerous official server sites worldwide which are already available
for public use, but more are always welcome. If you wish to run one yourself,
this section will explain how. The server also comes in handy if you are
behind a firewall and can't access any of the public servers. You can simply
run your own private server behind your firewall, and it will be accessible
from within your local network.

Once you have compiled and installed the server as described in previous
sections, there is little left for you to do. You simply need to arrange
for the server to run when a user attempts to connect to it. There are
two ways of going about this. You can have the server run out of inetd.conf,
or you can run the server in standalone mode.

To run the server as an inetd service, you (or your administrator) need
to put entries for cddbp into /etc/services and /etc/inetd.conf.
The format of these files may vary from platform to platform, but I have
included a fairly generic example of the necessary lines below.

The /etc/services entry might look like this:

cddbp		888/tcp			# CDDB server

Note that the port is "888". By convention, this is the port that cddbd
should reside at. If you do not assign this port to cddbd, clients that
wish to access the database on your system may be unable to connect to
the server.

The /etc/inetd.conf line might look like this:

# CDDB server
cddbp   stream  tcp     nowait  root    /usr/local/bin/cddbd cddbd

Note that this example assumes that the server resides in /usr/local/bin.
You should modify your version of this line to match the actual location
of the binary on your system.

To run the server standalone, simply execute it in the following manner:

cddbd -s default

This will cause the server to run in the background and listen at port 888
for user connections. If you wish the server to listen at a different port,
you can substitute that port number for the word "default" in the example
above. You may need to do this if you're not running the server with root
permissions, as port numbers below 1024 are usually privileged. Most clients
do support the ability to access the server at a nonstandard port, but need
to be configured to do so.

In standalone mode, the server will run until killed. It's best to use SIGHUP
if you wish to forcefully kill the server.

Note that it may be a good idea to set up an entry in /etc/services
for cddbp, regardless of how you run the server. It will look in this file
when determining what the default port to use is. It has a built-in default
of 888 that is used if /etc/services has no cddbp entry, so this may not be
necessary.

After the daemon is configured and ready to go, you should probably test
things out. The simplest way to make sure it's answering is to telnet
to port 888 on the system where the server is running and look for the
sign-on banner. It looks something like this:

201 cddb.moonsoft.com CDDBP server v1.3PL0 ready at Mon Nov 04 20:31:25 1996

This signifies that it's ready to accept commands. To test whether or not
it is really answering, try typing "stat" followed by the "enter" key. It
should print out various server statistics. The content of the output
is not important. What's important is that you see data, with a "." on the
last line by itself. This indicates that it is running properly and is
capable of accepting commands and returning data to the user.

The acid test is to actually use the server to download database entries
to a CD player client program. There are several of these. (See the README
file for a list.) Since you are installing the server on a Unix system,
it's likely that you'll want to run "xmcd", the de facto standard Unix CD
player client.

Once the CD player software is installed and running, follow the instructions
in the CD player distribution package which detail how to configure it
to query remote servers. Be sure to configure the client program to query
the system on which you installed the server. Once it's all running, insert
the CD of your choice in your CD ROM drive and do whatever is needed by the
application to initiate a query. With xmcd, for example, this is automatic
so you need only wait for a short time after inserting a CD. Other applications
may require you to click on a button to initiate a database server query.

If the entry is found in the database, your application should notify you
in some way that a match was found for your CD. Not all applications will
tell you if there was no match found, however. For absolute verification,
check the daemon's log file. Unless you disabled logging during installation,
every server access is logged, so you should see an entry made each time you
access the server.

Please note:

- DO NOT run the server on multiple systems which use the same NFS-mounted
cddbd work directory. This WILL NOT WORK, as the server implements filesystem
locks that don't work if shared by more than one system. It is okay if
the work directory is NFS-mounted, as long as it is guaranteed that one
system and only one system will be running the server. If you want to
run the server under these conditions, create a unique work directory for
each system and compile a copy of the server for each system with the
appropriate access file pathname.

Also, DO NOT run multiple copies of the server on separate systems that
share an NFS-mounted copy of the database. You can do this if necessary,
but it precludes configuring the server to accept automatic updates or
as a master hub. It also precludes the use of the "-C" option, or any
other server mode that modifies the database.

- If you wish to (re)build the fuzzy matching hash file at any time run the
server with the "-f" option:

cddbd -fdv

The "d" and "v" options put the server in debug and verbose mode respectively.
Debug mode causes output to go to stdout rather than the log file. If you want
info from the build to go the the log file, you can skip the "d" option.
Building the hash file may take some time, depending on your current system
load and how fast your system's CPU and disk are. You only need to do this
after installing the server, after installing a new version of the CD database,
or after performing a database update (see the section on automatic updates
below).

- You may notice that the log file grows as the server is used. Unlike most
Unix system log files, this one does not grow indefinitely and needn't be
monitored and truncated manually as it grows. The cddbd log is a circular
file that is kept at a constant size, as defined in the access file (see
the section on the access file below).


Section 3.2: Configuring the Daemon to Accept Automatic Updates
---------------------------------------------------------------

The publically available cddb distribution contains over 21000 entries at
the time of this release. However, the database is constantly growing on
a daily basis as new entries are submitted by dedicated users. There are
numerous "official" server sites worldwide that are fed new entries each
day as they arrive and are entered into the master database. These
official sites are maintained for public use, and suffice for all users
worldwide at this time. If you wish to run such a site however, whether for
public or private use, you are are welcome to. It can be arranged for your
site to receive daily updates from the master database via the Internet or
email.

If you wish to receive updates from the master database, you must contact
me for the feed to be initiated. Once the feed is started, you will receive
daily updates. For your system to accept updates, however, you will need
to set up cddbd to allow writes from the master site.

The most important step is to add a line to your access file to allow
connections from the site that will be feeding you new entries. While
any site can feed any other site, the only site I know of that actually
does this at this time is the master hub at "cddb.moonsoft.com". If you
wanted, you could set up your own master/slave hierarchy, but moonsoft.com
would have to be at the end of the chain somewhere! (Unless you come up
with a source of original cddb entries yourself. :)

If you answered "yes" during installation as to whether you wanted automatic
updates, you are already set. If you answered no, or didn't choose
moonsoft.com as your master site, you can either reinstall the server (no
need to remove old server files - just install over the current installation),
or edit the access file by hand. If you choose to do it by hand, you
will need to add a "permissions" entry for "*.moonsoft.com" with "connect",
"post", "update" and "get" specified. It may be that your site doesn't
recognize moonsoft.com, which may be due to the fact that there are several
aliases for moonsoft. If you have trouble, send me email. I suggest just
using the install script to save the hassle.

That's all you need to do. There are several ways of verifying that it all
works. If you configure your "logging" access parameter to log updates with
the "post" option, the daemon will log all updates. You can note the update
traffic with the "log day" server command. To see the total database count,
you may use the "stat" server command.


Section 3.3: Configuring the Daemon to Accept Email Submissions
---------------------------------------------------------------

This mode is used when accepting email submissions to the database. This
method yields the same results as direct reception (described in the
previous section), though entries are sent and received via email. This
is useful if your site is behind a firewall and can't accept incoming
connections, or if you and/or your upstream "master" site are not on a
connected network.

To configure the server to run in email mode, I suggest you (re)install
the server and answer yes when asked if you will be running the server
as a master hub. The server was designed to accept email submissions
when running as a master, though it works just fine if you are running a
slave site. Future versions of the install script will provide the
choice for you to accept email submissions without making you answer
yes to this question.

Answer the SMTP host and email address questions. The email address
is the name that will be put into rejected submissions, so it should
specify a valid address at which you can be reached. This address will
also be used to email rejected submissions to. If you don't want to
see these, remove the "bounce_email" access file parameter.

If you are running a master hub, you're set so far. If not, you need to
modify your access file at this point. Change the "dup_policy" parameter
to "always", the "postdir" parameter to specify the same pathname
as your "cddbdir" paramter.

Once this is done, you simply need to set up a mail alias. If you don't
have this capability, your system administrator may need to set one up
for you. How this is done varies from system to system, but in essence
you need to set up some email address which, if mailed to, will pipe
that email to "cddbd -m". On my system, the alias file entry looks
like this: 

cddb-mailer:	"| /usr/local/bin/cddbd -m"

Once an alias is arranged, you're all set. You should test it out by
mailing yourself a database entry to see what happens. Send yourself
both a good and bad one to verify that the correct thing happens.

Be warned: this method of accepting submissions is not secure. Unlike
direct transmission of new entries, anyone can send in new submissions
and the server will not discriminate. As long as an entry is valid, it
will be accepted regardless of the sender. This mode of operation was
really intended for people to submit new entries to the master database
itself, but serves as an alternate method of receiving new entries for
those limited to UUCP or stuck behind firewalls. Just be aware of the
security issues and don't blame me if your database is compromised in
some way. While this is not likely, it could be a problem. Possible damage
would be limited to the database directory itself, however.


Section 3.4: Configuring the Daemon to Transmit Entries to Remote Sites
-----------------------------------------------------------------------

The daemon is capable of transmitting database entries to other cddbd
servers, and to individual users via email. The current model of
transmission only allows one-way updates of new entries, so transmissions
are only made from database "master sites" to "slave sites". For example,
the official database master site is currently at amb.org. Entries
are submitted to that site by CD player applications via email, where
they are processed and added to the main database. New entries are then
transmitted to cddb.moonsoft.com where they are spooled and transmitted
to the 11 or so worldwide servers every day. Once a user submits an entry,
it ends up on all sites within a day.

In essence, a master hub is simply one that is both a "downstream" site and
and "upstream" site. I.e., one that receives new entries and then retransmits
them to other sites. Should any server in the flow be configured to transmit
entries back to one of its upstream sites, a transmission loop is created
that, while probably not harmful, is wasteful of bandwidth.

To configure your site as a "master hub" that transmits entries to other
sites, there are a few things that must be set up. First, you must be sure
that your site is set up to receive new database entries via email or
direct transmission as outlined in the previous two sections. Without some
sort of input for new submissions, you'll have nothing to transmit!

The most expedient way of setting up your daemon to transmit entries is to
(re)install the daemon and answer "yes" when asked if you wish to configure
the server as a master hub. Answer the questions regarding SMTP and your
return address, as these will be needed if you will transmit entries via
email. The install script will set up your access parameters appropriately,
although you can edit the necessary access parameters by hand if you wish
to avoid reinstalling (it's not that bad, just do it!)

To do this by hand, make sure you have a valid "postdir" defined. This
MUST NOT be the same as the "cddbdir"! In order for the server to
process and transmit entries, they must be spooled in a separate directory
hierarchy. The server will create the hierarchy as needed, as long as you
specify a valid path to the top level. You must also specify "dup_policy"
as either "compare" or "always". I suggest "compare" for this scenario, though
"always" will work if you won't be accepting email submissions.
The remaining parameters are: "transmits" set to 4; "smtphost" set to your
SMTP host (probably localhost); "admin_email" set to your full email address;
"logging" set to "errors hello access info post"; "dupdir" either blank or
not appearing in the file. Alternately, you may set "dupdir" to some hierarchy
other than "postdir" or "cddbdir" if you wish to preserve duplicate entries.
If you do this, duplicate submissions will not be merged with the database,
but will be saved in the directory hierarchy you specify. You will have to
remove duplicates by hand when you are through with them.

Once you've configured the daemon for to run as a hub, you need to set up
a few simple timed scripts to initiate the update and transmit process.
The command:

cddbd -uft all

will initiate the update, rebuild the fuzzy hash file to include any
new database entries, and transmit new entries to all downstream sites.
This process involves examining new entries in the postdir, merging them
with the database and noting them in the history file, and conversing with
remote sites. This can be somewhat lengthy, so if you wish you may exclude
the "f" option. This will skip building the fuzzy matching hash file, and
will speed up the process immensely. You don't need to update the hash file
every time, so this might be a good thing to skip if your system is adversely
affected by the rebuild.

I suggest setting this up with cron to run once a day, or more if your system
administrator doesn't mind the disk activity. This also causes a lot of
activity on the remote sites, so you might check with the administrators
on your downstream end to determine the best times and frequencies for
transmits. Since remote sites or networks can be down, you should probably
also run the following command at least several times a day:

cddbd -t all

This will catch up any remote sites that might have been unavailable at the
time of the update. This command simply transmits files, which is inexpensive,
so it can't hurt to do this often.

Alternately, if you wish to transmit to an individual site, you can use:

cddbd -t sitename

where "sitename" is the name of the site to transmit to. This is useful if
certain sites have restrictions on when you can transmit, and you want to 
set up a special cron job for transmission to that site. There is no timed
mechanism built into cddbd - you must manage it through the use of cron.
In general, you should only require the initial update command once a day,
and a few lone transmits scattered throughout the day.

To set up transmission to a new site, you will need to create an entry
in the "sites" file for that site. There are several forms of entries
in the sites file, but there are only two for the purpose of transmitting
entries:

site.domain.com  cddbp [port]
site.domain.com  smtp  user

The first is for sites that you will be transmitting entries to via the
CDDBP protocol across a network. The port argument is optional, and is
only specified if the remote server is at a nonstandard port. The second
is for sites that you will be transmitting entries to via the SMTP protocol.
The "user" argument is not optional, and specifies the email address or alias
that you will be sending entries to at that remote site.

When you add a site to the sites file for transmissions, it will be
transmitted to the next time you execute the server with the "t" option.
All new files after the next update (the "-u" option) will be sent to that
site. If you wish to retroactively schedule the transmission of older
entries, use the "-T" option. This takes a short while to run, and
searches the database for files with time stamps within the specified
range. It updates the history for that site appropriately, so the relevant
files will be scheduled for transmission to that site. See the sections
below on command line options and the sites file for more information.

If you wish to check the status of transmissions, you can use the "stat"
server command. It will display the current transmission status of all
downstream sites. Also, you should periodically check the log file for
transmit errors and other problems that might hamper file transmission.

If you decide to stop feeding entries to a remote site, simply remove
or comment out the entry for that site in the sites file. The history
for that site will be automatically cleaned up the next time you transmit
to any site.

If you wish to look at the log statistics for a remote site, you can use
the following command:

cddbd -l all

or 

cddbd -l sitename

This will cause the server to download the log stats from the relevant
server(s) if you use CDDBP to talk to them. Sites that you talk to with
SMTP cannot be contacted in this fashion. This is useful for getting
information on usage and errors on the sites you feed. If you are having
problems with a remote site, it is often useful to download the log from
that site. You can do so by connecting to that site (via telnet) and
executing the "log get" command. This will cause the remote server to dump
the log in its entirety. You might want to have some sort of capture program
such as "script" running when you do this, as it's likely to be lengthy.

If at any time you wish to verify the integrity of the database, you can run:

cddbd -cdv

This checks the database from top to bottom, verifying the validity of each
entry. This takes about as long as building the fuzzy matching hash file, so
sit back and relax. If this finds some errors with links or permissions, you
may want to repair things. Use the following command to do so:

cddbd -Cdv

This does the same thing as the former command, but repairs broken links and
resets file permissions. It may not be able to repair everything, as some
things require hand-maintenance. Future versions of the server will fix
everything short of blatantly illegal database entries, so for now if there
are still some errors after cleaning the database you should just ignore
them. Database irregularities are not a cause for alarm, as they won't
adversely affect the normal operation of the server.

Checking the database once in a while is a good idea, just to be sure that
all is well.

Please note:

- DO NOT interrupt the daemon or its children while it is updating the
database or transmitting. This may cause you to lose entries. Great
precautions have been taken to ensure database integrity where possible, but
it is still possible to cause problems if you kill the daemon when it's
modifying the database.

- Sites that you feed entries to must have the proper permissions set to
allow your site to post new entries, if you use CDDBP. Without the proper
permissions, transmission will fail.



Section 4. How Clients Should Use the Server
--------------------------------------------

If you are interested in creating an application that uses the server, this
section provides a few hints and tricks to help you out. More detailed
information than that provided here can be found in the CDDBPROTO and DBFORMAT
files included in this package. There is also an excellent document describing
the nuts and bolts of using the entire cddb/cddbd package called "cddb.howto"
that can be found at ftp:ftp.moonsoft.com/pub/cddbd-1.3.1/cddb.howto.gz.

In general, clients will wish to acquire a database entry for a particular
CD that a user has inserted into his CD ROM drive. By specifying the proper
parameters to the "cddb query" command the server will attempt to locate the
proper database entry for the user. The user client may download an entry
located in this fashion using the "cddb read" command.

There is little else most clients will ever need to do, other than signing
in using the "cddb hello" command. There are some pitfalls and intended uses
of the server that clients should be aware of, however:

Some clients will perform a second query if they are presented with a fuzzy
match for a particular CD. This is unnecessary, as the client can simply
perform a read on any fuzzy match located by the server.

When signing in using the "cddb hello" command, the client should specify
the actual user name and system name of the person running the client. Some
clients always specify the same hard-coded information regardless of the
actual user and system name. This is discouraged because user information
is used when gathering statistics on server usage, and incorrect hello
information can affect these statistics. Some operating systems, such as
Windows 95, may not have a notion of a user name. Clients that operate
under such conditions should allow some way for a user to specify his or her
user name.


Section 5. Getting Help
-----------------------

If you have questions on any other topic related to cddbd, please send email
to:

steve@moonsoft.com

I am more than willing to answer any question and provide assistance. You can
also contact me at this address if you wish a database feed of your own.


Appendix A: The Access File
---------------------------

The install script sets up cddbd with simple defaults that should work
in most cases, assuming you provided the config and install scripts
with correct pathnames. However, cddbd is highly configurable and its
behavior may be modified by making changes to the access file. Depending
on your desires, you may wish to alter the defaults to tailor the server
to your meet your requirements.

The behavior of the daemon is determined by the contents of the access file.
By setting parameters in this file in the appropriate manner, all aspects of
the daemon's behavior can be controlled. There are numerous parameters,
and you should not have to worry about most of them. The parameters and what
they signify are outlined here.

All parameters should each appear on a separate line. Numerical options
should be a positive decimal value. Pathname arguments should always specify
absolute paths.

The "logfile" should appear first so errors in the file will get logged.


cddbdir:
	Absolute pathname to the directory where the CD database is located.
	This is the only required field in the access file.

logfile:
	Absolute pathname to the log file. This should probably reside in
	the same directory as the access file, but doesn't necessarily
	have to. This should appear as the first field in the access file
	in order to ensure that syntax errors in subsequent entries in
	the access file get logged.
 
motdfile:
	Absolute pathname to a file containing the "message of the day".
	This should probably reside in the same directory as the access file,
	but doesn't necessarily have to. The contents of this file will
	be displayed to the client when the "motd" server command is executed.

sitefile:
	Absolute pathname to a file containing information on remote sites.
	This information is used when transmitting database entries,
	and as data for the "sites" command.

histfile:
	Absolute pathname to the transmit history file. This file is used
	when updating the database and transmitting to remote sites.
 
hashdir:
	Absolute pathname to the directory for hashing index files. This
	is where the fuzzy matching index goes, so this directory must
	be specified in order to enable fuzzy matching.

postdir:
	Absolute pathname to the directory for posting new entries.
	This should be the same as cddbdir if you running a slave site.
	Since by enabling posting you are allowing others to write
	to your system, you might want to make this a directory somewhere
	in your spool directory hierarchy in order to avoid having a
	malicious user clog up your filesystem. Be sure to set up your
	posting permissions correctly in any case.

	Note that no clients to date support the posting of new entries to
	the server. Currently, the preferred way to do this is to send
	new entries via email to xmcd-cddb@amb.org. This feature is
	primarily for the purpose of updating the database via a remote
	cddbd master site.

dupdir:
	Absolute pathname to the directory for putting duplicate entries.
	This directory is important if you are maintaining a master copy
	of the database. Files in the post directory that are duplicates
	of entries in the database will end up here when an update is run.
	Duplicates must be dealt with by hand; whether that means deletion
	or hand-inspection is up to the database administrator. If no
	path is specified, duplicates will be deleted automatically.

lockdir:
	Absolute pathname to the directory for locking.
	This directory should be writable by the uid that the server will
	be running as. This directory should probably reside in the same
	directory as the acces file, but doesn't necessarily have to.

input_time:
	Input timeout value in seconds. User is disconnected if idle
	longer than the timeout. A zero value disables this feature.

	Should be less than or equal to "connect_time".

access_time:
	Access timeout value in seconds. User is disconnected if
	a database access attempt is not performed within the allotted
	time. A zero value disables this feature.

	This keeps users from connecting via telnet and staying connected
	without actually using the server.

	Should be greater than or equal to "input_time" and less than or
	equal to "connect_time".

connect_time:
	Connect timeout value in seconds. User is disconnected after
	being connected for the allotted time. A zero value disables
	this feature.

	This keeps the user from hogging the server for too long, regardless
	of how he uses it.

	Should be greater than or equal to "access_time".

xmit_time:
	Input timeout value in seconds. Connections to remote sites
	are terminated if transmissions stall for longer than the
	timeout. A zero value disables this feature.


fuzzy_factor:
	The frame count factor used in comparing track offsets for
	fuzzy matching. This is the number of frames that any one track
	can be off by. Frames are 1/75 of a second.

	This should probably be left alone. Tuning this upward makes for
	a more liberal matching algorithm that is prone to find incorrect
	matches, and tuning it downward makes for a stricter matching
	algorithm that is more likely to miss legitimate matches.

fuzzy_div:
	When divided into "fuzzy factor", this is the average number of
	frames track offsets can differ by for fuzzy matching.

	This should probably be left alone. Tuning this downward makes for
	a more liberal matching algorithm that is prone to find incorrect
	matches, and tuning it upward makes for a stricter matching
	algorithm that is more likely to miss legitimate matches.

logging:
	Logging verbosity flags. If logfile is not specified, logging
	is always disabled, regardless of this setting. One or more of the
	following should be specified separated by white space:

	none:	Logging disabled. This nullifies all other log flags.
	hello:	Logs "hello" data. Useful for keeping track of database "hits".
	access:	Logs database accesses.
	post:	Logs database postings.
	info:	Logs general info.
	input:	Logs all user input.
	errors:	Logs program errors.
	all:	Enables all logging flags.

dup_policy:
	The value of this variable dictates wheter duplicate entries are to
	be copied over existing database entries. This variable may be
	specified with one of the following options:

	never:   Never copy over an existing database entry with a duplicate.
	compare: Compare duplicates with existing entries before copying over.
	always:  Always copy duplicate entries over existing entries.

	Duplicates that are rejected for copying are put into "dupdir" if
	it exists, otherwise they are deleted. It is suggested that
	slave sites set this to "always" and master sites set this to
	"compare". If not specified, the default setting is "compare".

transmits:
	The value of this variable specifies how many sites can be
	transmitted to simultaneously. Since transmitting is inexpensive,
	it is safe to set this number high.

smtphost:
	The hostname of the system to contact when sending mail via SMTP.
	The most likely host is "localhost", but someone may want to
	set this to another address.

admin_email:
	The full email address of the daemon administrator. This is used
	as the return address when the server sends email.

bounce_email:
	The full email address to which copies of bounced email submissions
	should go (in addition to the original sender). This should
	generally be set to those who are interested in debugging problems
	with submissions from client applications. If this parameter is
	empty or missing, only the originator is mailed when a submission
	is rejected.

log_hiwat:
	The maximum size in bytes the log file is allowed to be. If the log
	file should increase in size so that it exceeds the high water mark,
	it is truncated.

log_lowat:
	The size the log file is truncated to when it grows to log_hiwat
	bytes in size.

post_lines:
	The max number of allowable lines a posted database entry
	may be. A line is currently limited to 256 bytes.

users:
	The max number of users that can access the database at once.
	If set to zero, no limit is placed on the number of
	simultaneous users. Accesses to the database tend to be
	very quick and efficient, so very little additional load is
	placed on even a slow system by the server. You might want to
	consider disabling this feature or setting it to a very high value,
	unless your system is an extremely busy one.

user:
	The user name of the user that owns the daemon files. This may
	also be a numeric user ID. If the word "default" is specified,
	the user name of the invoking user will be used.

group:
	The group name of the user that owns the daemon files. This may
	be the name of any group that the user belongs to. This may
	also be a numeric group ID. If the word "default" is specified,
	the group name of the invoking user will be used.

file_mode:
	The file permissions, in octal, that server files should be set to.

dir_mode:
	The file permissions, in octal, that server directories should be
	set to.

db_user:
	The user name of the user that owns the database files. This may
	also be a numeric user ID. If the word "default" is specified,
	the user name of the invoking user will be used.

db_group:
	The group name of the user that owns the database files. This may
	be the name of any group that the user belongs to. This may
	also be a numeric group ID. If the word "default" is specified,
	the group name of the invoking user will be used.

db_file_mode:
	The file permissions, in octal, that database files should be set to.

db_dir_mode:
	The file permissions, in octal, that database directories should be
	set to.

permissions:
	Whether to allow/disallow connections, posting entries to "postdir"
	via "cddb write", whether to allow remotely initiated database
	updates via "update", and whether to allow the user to get the log
	and other files. There may be multiple of these lines in the file
	in order to specify different permissions for various hosts.

	The format is:

	permissions: remote_host connect post update get

	The arguments are:

	remote_host: An Internet hostname, network name, domain name,
	    IP address or the word "default" for all hosts. Any
	    host matching this field is given privileges as specified
	    by the following fields. Only one default line may
	    appear in the file. If a host matches more than one
	    specified permission, the most specific match is used.
	connect:
	    Valid values are either "connect" or "noconnect" to
	    either allow or disallow connections.
	post:     Valid values are either "post" or "nopost" to either
	    allow or disallow posting.
	update:   Valid values are either "update" or "noupdate" to either
	    allow or disallow remotely initiated updates.
	get:      Valid values are either "get" or "noget" to either
	    allow or disallow remote acquisition of the log and other files.


Appendix B: The Sites File
--------------------------

This file contains the hostname and/or email addresses of
remote database servers that are fed database entries by your
site, or that are queried for log information.

Remote site entries may take four forms:

	site.domain.com  cddbp [port]
	site.domain.com  log   [port]
	site.domain.com  smtp  user
	site.domain.com  info  <port | -> latitude longitude description

The first form specifies an Internet host address that can be reached
with CDDBP for the purpose of uploading new database entries.
The "port" argument is optional, and if it is not specified then
the default port will be looked up in /etc/services. Failing that,
a built-in default will be used. The port number should not be
necessary unless the cddb server being accessed is at a nonstandard
port. Example:

	cddb.moonsoft.com cddbp 888

The second form specifies an Internet host address that can be reached
with CDDBP for the purpose of acquiring log statistics. The "port"
argument is optional, and if it is not specified then the default
port will be looked up in /etc/services. Failing that, a built-in
default will be used. Example:

	cddb.moonsoft.com log 888

The third form specifies an email address that new database entries
will be emailed to via SMTP. The "user" argument will be combined
with the site name to form the email address. Example:

	cddb.moonsoft.com smtp steve

This causes mail to be sent to steve@cddb.moonsoft.com.

The fourth form specifies location information for a known Internet
server site. This information is displayed as output with the "sites"
server command. The "port" argument specifies the port that the
server resides at on the remote site; a "-" may be specified if the
remote server resides at the default port. The "latitude" and
"longitude" arguments specify the global coordinates of the server
site. These should be specified in the following format:

	CDDD.MM

where "C" is the compass direction (N, E, W, S), "DDD" is the
degrees, and "MM" is the minutes. The "description" is an
arbitrary length string describing the site. Example:

	cddb.moonsoft.com info - N037.23 W122.01 Moonsoft, Sunnyvale, CA USA

It is allowed to have up to one of each form for the same site.


Appendix C: Other Files
-----------------------

There are several other important files used by the server.

The log file:

The log file is important, and you should make it as large as you reasonably
can (with the "log_hiwat" and "log_lowat" access parameters).  The daemon
notes almost every aspect of its activity in the log file, and it
can be crucial for figuring out problems with operation. Also, the log
file is used to keep daemon usage and error statistics. You can view
these with the "log" server command. You can download the log file from
remote sites with the "log day" command, if permissions on that site allow it.


The history file:

This file, though in ascii format, is not intended to be modified by hand.
The server uses this file to maintain a log of outstanding files that have
yet to be transmitted to remote sites.


The fuzzy matching hash file:

This file is created by the server with the "-f" option. It contains a
sorted list of database entries and some important statistics necessary
for quickly finding fuzzy matches for a user query. The server will operate
if this file doesn't exist, but it will complain profusely.


The lock directory:

This directory is where the daemon puts file locks. It should be left
untouched by the user.


Appendix D: Server Options
--------------------------

The CD database server has several run-time options that you can use to
override the default behavior for debugging purposes. There are three
usage modes:

cddbd <-m|-M> [-dqv] [-a access_file]
cddbd -s <port|"default"> [-dqv] [-a access_file]
cddbd [-Cc <level|"default">] [-uf] [-l|t <rhost|"all">> |
    <-T <rhost|"all"> <hh[mm[ss[MM[DD[YY]]]]]>] [-dqv] [-a access_file]


Except as noted below, options for one mode cannot be mixed with that of
another mode.


The following options are valid in conjunction with any other combination
of options:

The "-d" option puts the server in debug mode. All log output is routed to
stderr rather than the log file when this option is invoked. This is useful
for debugging when the server can't find the log file for some reason.

The "-v" option puts cddbd in verbose mode. This causes cddbd to generate
more verbose log output.

The "-q" option puts cddbd in quiet mode. All log output is suppressed
when this option is invoked. This option nullifies the "d" and "v"
options.

The "-a" option forces cddbd to use the specified pathname as the location
of the access file regardless of the path that was compiled into the server.
This is useful when cddbd can't find the access file or when you want to
test out an alternate access file without disturbing the installed version.


The following options are used with the daemon mail filter mode:

The "-m" option puts cddbd in mail filter mode. In this mode, it expects
one full email database submission, complete with mail header, to be fed in
through standard input. The entry will be processed and placed in the "post"
directory for later inclusion in the database, if it is valid.

The "-M" works much the same as the "-m" option, but does not actually
post submitted entries. Instead, confirmation is echoed to the sender if
the submission is acceptable. This mode is useful for testing client
applications that wish to develop and test new entry submission capability.


The following options are used with the server standalone mode:

The "-s" option puts cddbd in standalone server mode. It forks and becomes
a background process, listening at the specified port number for connections.
If the word "default" is specified instead of a port number, then the
daemon uses the most appropriate default. It will first check /etc/services
for a port number, and failing that, will use the built-in default of port
888.


The following options are used with the various daemon database management
modes:

The "-c" option causes cddbd to check the integrity of the database.

The "-C" option causes cddbd to check the integrity of the database,
and correct what problems it can. This option supersedes the "c" option.

Both the "-c" and "-C" arguments require a numerical argument in the range
(1, 2), inclusive, which corresponds to the level of strictness to be
employed when verifying database entries. Level 1 causes normal checking to
be performed, while level 2 causes more paranoid (and possibly incorrect)
rules to be applied. The latter is useful for locating entries that have
dubious characteristics but may actually be valid. Using "-C" at level
2 is not recommended.

The "-u" option causes cddbd to initiate an update of the database with
new entries from the "post" directory. It updates the transmit history
file for remote sites.

The "-f" option causes cddbd to build the fuzzy matching hash index, then
exit. This option is best used in conjunction with "-d", otherwise
any errors encountered during the creation of the hash file will go to
the error log.

The "-l" option causes cddbd to obtain log status information for the named
remote site, or all remote sites if the word "all" is specified.

The "-t" option causes cddbd to transmit all outstanding new database
entries to the specified remote site, or all remote sites if the word
"all" is specified.

The "-T" option causes cddbd to reset the transmit history for the named
remote site, or all remote sites if the word "all" is specified. The history
is reset to the specified time, or to the current time if the word "now"
is specified instead. Any portion of the time that is not explicitly specified
will be substituted with the current value of that portion of the time.

If multiple database management options are invoked at once, the server
runs them in this order: Ccuf<l|t|T>
