# Cryptix Developers Team - FAQ

Revision is $Id: FAQ.html,v 1.13 1999/07/26 15:46:08 iang Exp $

Table of Contents

- Description and Information

- Introduction
- Mailing List
- Business Model
- Admin
- The Cryptix Homepage
- To Do

- Source Access

- Introduction
- Access - Unix
- Access - MS
- Some Notes
- Usage
- Resources
- Posting Progress
- Creating a New Repository

- Work In Progress

- Current Active Projects
- Wish List
- Next Major Release

- Distribution

- Introduction
- Tagging
- Building the Distribution
- Adding New Code
- Some Other Comments

- Programming Standards

- Security
- Getting the Best out of Cryptix
- Unit Tests
- Documentation
- Applets
- Whitespace
- Final Keyword
- File Name Conventions

- Politics, Culture, etc

- How is the team organised?
- Why is the mailgroup "by invitation?"
- How can I help someone into the team?
- Culture
- A Rant on the License

- Team-FAQ History

Return to the Cryptix Index. Headings below return to the above table of contents.

## Description and Information

### Introduction

The Cryptix Development Team develops and publishes new versions of Cryptix (primarily in Java, Perl is somewhat quiet).

### Mailing List

A mailing list exists for the team. This list can be high volume. It is currently by invite only, as the original developers are protective of a private environment for the team. This could change. Mail the list maintainer for details.

Please don't post large source to this group. Send it to the intended recipient directly. As the state of the lists is such that individuals are keeping their own archives, sticking source in doesn't help.

Ian has set up a wonderful archive . No more humungous mailboxes !

### Business Model

The team is a volunteer group. No funds are currently provided.

### Support

Support for team activities is on an ad-hoc basis.

### The Cryptix Homepage

Current Situation:

The Cryptix Homepage is a stalled at the moment. The important contents have been shifted to the distribution, and the Mirrors perform the normal functions of the homepage. Much of the old contents can be found in `guide/cryptix`.

We have the three cryptix.* domains. The issue is where and how to site them. One view is:

- cryptix.org internal home page for the team.
- cryptix.net mirror index, distribution issues.
- cryptix.com commercial page, expositions of projects, aimed at the professionals and companies who want something "serious."

To mount these domains costs money and/or somebody willing to host them...

### To Do

Mailgroups.

## Source Access

### Intro

Source is located in a Concurrent Version System (CVS) archive that is accessible over the net via Secure Shell (SSH).

Here is a good introduction to CVS. There are more resources listed below.

### Access - Unix

#### Software Needed

You will need both SSH (I use 1.2.20) and CVS (I use 1.9.10, it needs to be at least client-server enabled) installed on your system. It should be working.

#### Creating a Key

In order to access the archive, create an SSH key:

        $ ssh-keygen
        Initializing random number generator...
        Generating p:  ........++ (distance 100)
            blah blah ...
        Enter file in which to save the key (/home/me/.ssh/identity): cryptix-cvs
        Enter passphrase:                  <<=== remember this bit
        Enter the same passphrase again:
        Your identification has been saved in cryptix-cvs.
        Your public key is:
        1024 43 6425more numbers ...5623 me@crypto.hackers.com
        Your public key has been saved in cryptix-cvs.pub
        $

The public key is the long printed line of numbers terminating in your email address. It is also in the file `cryptix-cvs.pub` . Send this to the archive maintainer for adding into the CVS `authorized_keys` file. To avoid Mallory joining the party, send it PGP encrypted and signed.

Please don't use this key for any other access, create other keys for other purposes. See below for how to set up the key as a different name and wrap a script around it.

#### Environment

Next, add this to your environment:

        export CVSROOT=:ext:cryptix@helplets.com:/home/cryptix/cvsroot
        export CVS_RSH="/usr/local/bin/ssh"

in your .profile or equivalent. Notes:

- that you may need to use a different location for the ssh program such as `/usr/bin/ssh`.
- the location of the archive will change from time to time. If and when it does, you will probably have to start off with a new archive, as CVS stores details about the location of files in its CVS/ tree, which won't allow it to handle shifts. (Is this a bug in CVS?)
- The location of the archive at the server is needed by the client CVS software, but this is an artifact of the current CVS design, and will disappear one day. For the moment, the repositories are located in `cvsroot` within the `/home/cryptix` account.

#### Testing

Now, try the following:

        cvs get README

which will explain some archive details (not really written yet, still a test file). Once you get that file, you have proven access, so can go onto the next step, Usage, below.

#### Making SSH jump through hoops

You may need to convince SSH to do some special things like:

- Go through a gateway that is on the net, when you are on a machine that is not on the net. SSH doesn't handle proxies (they're working on it apparently).
- use a different identity such as `cryptix-cvs`, above

To modify the SSH access from CVS is a little tricky. I wrote a script that pretended to be ssh, called ssh_helplets:

        #!/usr/local/bin/bash

        # set -x                                    # uncomment for debugging

        ssh_gateway="ssh gateway.org"               # set this to null if no gateway

        ssh_identity="-i .ssh/helplets"             # use this key only for helplets
                                                    # identity file is at the gate

        exec ${ssh_gateway} ssh ${ssh_identity} $*

Then, I set the CVS_RSH (above) to call this program directly, and created the named key (above) on the firewall `gateway.org (the files `helplets*` are located in the `.ssh` directory).

Once set up, when run, you should see something like (with set -x uncommented):

        me$ cvs get README
        + ssh_gateway=ssh gateway.org
        + ssh_identity=-i /home/me/.ssh/helplets
        + exec ssh gateway.org ssh -i /home/me/.ssh/helplets helplets.com -l cryptix cvs server
        Host key not found from the list of known hosts.
        Are you sure you want to continue connecting (yes/no)? yes
        Host 'helplets.com' added to the list of known hosts.
        cryptix's password:
        Permission denied.
        cvs [checkout aborted]: end of file from server (consult above messages if any)

At which point it is time to send the new key.

Note that to setup an ssh through a gateway, you need account access to the gateway. SSH currently doesn't support proxy access directly, but it is being added.

### Access - MS

Here are Ian's instructions for MS platforms based on his experiences with NT. They should work for W95. Please send any experiences to the list.

#### Software Needed

Download one of these.

- Also try Putty which is UK stuff. It doesn't do port forwarding though.

- From NTK (except readers say Putty, above, is better):

                                 >> TRACKING <<
                          look what the mouse dragged in

               Until recently, if you wanted a Windows secure shell
               connection to your Unix box, correct medicinal procedure was
               to download the official Data Fellows software, use it until
               the trial period expired, try to crack it, fail, then give
               up and install Linux. Nowadays, even that doesn't work,
               because those same fellows of Data are trying to encourage
               everyone to upgrade to their new SSH2 protocol (special
               powers include ultra-secure complete-incompatibility-with-
               the-ssh-your-Unix-box-uses mode), and only the most
               outrageous warez dudes can find their original, working,
               software. Thus, the sudden renaissance of TERATERM PRO, an
               ancient Windows terminal program that has the singular
               advantage of having an ssh plug-in. Okay, it's not new,
               exactly, but one day soon you'll need to find it, and
               careful observers will note that no new software has been
               invented since Netscape 1.1N. And even that was just
               Hypercard with a glowing 'N', right?
               http://www.zip.com.au/~roca/ttssh.html
                   - caution, Americans! RSA don't want our stinking foreign
                                                            code in your PCs
               http://hp.vector.co.jp/authors/VA002416/teraterm.html
                             - Japan, too. Smacks of World Government to me.

- Sascha put together this one-stop-shopping archive from these sources:

  For cvs: I downloaded the normal cvs-1.9.16.tar.gz and fixed a bug on Win32 when using an external RSH program. The source can be requested from me. The patch (for cvs-1.9.10) is available on the same site that has the ssh binaries.

  For ssh: I downloaded it from bmrc.berkeley.edu but the ssh-keygen does not work.

  There are other sites offering ssh for Win32: therapy and martin

  The version from bmrc.berkeley.edu worked best for me.

- SSH-keygen and SSH and CVS from Ian's homepage, or

- the same components or the one-stop-shopper collection from Set's homepage but note that Sascha reports connect problems and assertion failures with the `ssh.exe` there.

#### Creating a Key

Execute `SSH-keygen.exe` to create a key pair. Mail the contents of the public key file (by default, identity.pub), PGP encrypted and signed, to archive maintainer

Once the archive maintainer has received your e-mail with your new SSH public key, he will add it to his list of authorised users and reply to your message. You should now be able to run CVS correctly. Try executing cvs get README to check everything is working correctly.

#### Environment

Set the following environment variables (using System Properties Environment under NT, or by putting an entry in your Win95 autoexec.bat):

- CVS_RSH should be set to the full path to your SSH.EXE file (including the filename itself). Make sure the path does not include any directories with spaces in their names as this throws CVS.
- CVSROOT should be set to

          :ext:cryptix@helplets.com:/home/cryptix/cvsroot

#### Making SSH jump through hoops

Just as with Unix, it is possible to make SSH do tricks. In order to go through a proxy machine, assuming you cannot access the archive directly, you can point CVS_RSH to this batch file:

        @echo off
        ssh.exe -l login_name gateway.machine.com ssh -i .ssh/cryptix-cvs %1 %2 %3 %4 %5 %6

In this case, the first SSH will jump on to the gateway machine, and there, run another (Unix) SSH to connect to the repository.

Tips.

- In general, test SSH first, then test CVS.
- The gateway needs to have the key .ssh/cryptix-cvs in existance first, test this first from the gateway itself.
- Once the Gateway is working, test the batch script by running it:

              ssh.bat helplets.com pwd

- On MS, the SSH command needs to specify the remote identity, as there is no strong identity in an MS system.
- The batch script needs to emulate SSH, with all keys set (possibly by default), but without the final identity.
- Then test the CVS by doing a

              cvs get README

- To debug the batch script, edit out the `@echo off` line and look at the commands generated.

### Some Notes

Some comments:

- On W95/applets et al Raif said:
  There's a jCVS: a Java front end to UNIX CVS. Have a look at <http://www.ice.com/cvs/>. I dld it just in case it'll do the job ;-) I guess I'll know better once it's up and accessible.

- from mpp@mitekgold1.miteksys.com :
  ... we are using CVS with SSH. I think that this would work well from Unix hosts, but from PC's there isn't a good SSH client available. We have been using SSH from DataFellows and it has bugs that make it hard to use, especially on Windows 95. On 95 we see problems with data corruption which makes it really hard to use. On NT I think we only saw problems with uploading data to the server and we haven't seen corruption for a long time. There is also a public port of SSH 1.2.17 to Win 95 and we began to use it, so far nobody is reporting problems. This public port, though has a few limitations, one of each requires setting up a new SSH connection for every CVS transaction and typing in the password for every CVS transaction. Still, it's better than not working at all.

  Set's homepage

  You can also try www.datafellow.com - they have the commercial version, but it does not work from the command line and has bugs in port forwarding. The public port is the other way round - it works from command line but does not do port forwarding.

### Usage

The entire hierarchy can be seen by browsing at http://helplets.com/java/cryptix-cvs/.

Here is a quick list of CVS commands:

-          cvs get current

  gets the whole tree into cryptix/.... It's big, and this will takes some time.

           cvs get current/doc/AllNames.html

  will get that single file. Sub-directories work as well.

  Nota Bene. CVS gets the named tree into your current directory, not just the named file. That is, this will create a directory `current/doc` and put `AllNames.html` into it. The `get` command is relative to the top of the tree, not to your current position in your own tree. It catches me every time...

-          cvs update

  in any directory within the hierarchy (including the top) will bring down and patch in any newer versions. If there is a conflict, it will tell you, and tell you how to deal with it.

-          cvs commit [files...]

  will send your files in the current directory up to the CVS archive for incorporation.

-          cvs add files [...]

  will add new files from your current directory into the archive. For importation of new trees, look up `import` instead.

  Nota Bene for MS users. CVS will automatically convert the files from DOS format to Unix format. If you are adding a non-text file such as a GIF, then add the -kb option

           cvs add -kb files [...]

  to tell CVS to not add this file with all carriage returns stripped out.

The above commands should get you going, but be prepared to dive into the doco, see the resources below, when you want something beyond the basics. CVS is a big complex system, with many features and many traps for young players. That's another way of saying I haven't mastered it yet...

### Resources

This is a list of resources on CVS.

An Introduction to CVS
USDA CVS site has these resources:

- The CVS Story Book is a FAQ including lots of links to resources
- An archive of mail in months on main page
- mail search of posts
- A Mini FAQ on the CVS mail group

CVS tutorial
Cyclic are commercial supporters of CVS. They have an ftp repository .
The old (out of date) 1995 FAQ
OpenBSD's Anon CVS update page

### Posting Progress

Unless your task is completely obscure or silly, it is customary to post intentions, progress and completion notes to the mailgroup. This is so that others can help, flag problems, discuss issues and methods, or consult with you if they need to get into that area.

### Creating a New Repository

These notes are here because it's the logical place, rather than there being a need to create repositories often.

First thing is to make sure SSH and CVS are installed, as above. If these don't work, at least in client mode, then there is little hope for making a repository.

Next, make an account.

Next, set up an SSH directory and file with the authorized_keys in it of all the permitted accessors. An alternate way to do this is to leave off this file and distribute the password.

Finally, run the following command:

       cvs -d $PWD init

from within the account.

To access it, set up the CVS environment variables as described above.

There are more things you can do - set up chroot and only-cvs access, and make web archives, but I don't know how to do that yet.

## Work In Progress

### Current Active Projects

These are the current active projects.

Who | Project | Description | Status
Twente | SSH | V2. Twente Uni team in full is Jeroen Edwin, Erwin | old.
Sascha | SSL V2 | secure browser protocols, we need a snapshot release | 2nd release ?
Jeroen | IJCE | IJCE to spec V1.2 | WIP
Raif | AES | NIST page describes. | live LOKI'97, Serpent, Pentagon, kit published
Paulo | EC | ElGamal over Elliptic Curves, Square (with Raif) | Analysis
Ian (Quincy) | Enigma | secure mailer plugin for all mail handlers. Also looking at guerrilla groups and PFS | published in Cj2.2, bug fixing, also in ByProxy.
Edwin | PGP | produce complete compatible PGP in Java - JCE then OP. | WIP
Erwin | WebFunds | see webfunds.org. | works, distributes. Needs OpenPGP

Table 1: Current active projects

### Wish List

We want everything ... well, almost. Here's a current list of wants, there may be others:

- Hardware.
  - Oxford Hardware Compilation Group.
  - Eracom have a PCI board that David Hook is writing to.
  - nCipher have a SCSI device that JCP are writing to.

- Dr Dobbs article. Look at their author's notes. Also, there is the QuickStart notes.

- Mathematical algorithms. See for example algorithms in Chapters 2, 3, 4, 5, 14 in C from Pate Williams for the book Handbook of Applied Cryptography, Menezes, et al.

- DSA algorithm. We need the JCA algorithms for the free Java implementations that do not/can not use the JCA. So we probably also need the DSA sig algorithm. What others are in the SUN provider?

- P1363 standard. This is a new standard for public key cryptography that we want to be compatible with. Includes things like:
  - Rabin-Williams
  - elliptic curve schemes
  - OAEP encryption padding (an improvement over PKCS#1)
  - standard ASN.1 types for keys, etc.
  Their home page is here (username "P1363", password "MarsRoks").

- A course! Yes, I'd like to do an online, interactive course in applied cryptography. Something like:
  - coupled with Cryptix code: exercises, projects
  - written or guided by an academic name
  - a series of modules: number theory, cryptography, practical, protocols
  - along the style of the OU
  - assume programming and some mathematical ability.
  - exams? a certificate? a tutor mailgroup?
  Here's a list of existing meatspace courses.

- PKI - either pick up SPKI or extend Open PGP.

- PRNG - generation of random numbers. This area is always controversial, so could do with some attention:
  - analysis of random numbers generated by current methods
  - simple test programs to analyse results
  - new modules using different methods
  - documentation of issues, work done, results of analysis perhaps in a CAN.

- IPSec - whilst C IP stacks aren't really our area, there is some possibility that an all-Java kernel and IP stack will find a place in things like the NC or smart cards.

- The interesting and secure secret key algorithms in Schneier, accompanied by test and examples. E.g.,
  - GOST
  - 3-WAY
  - CA-1.1
  - FEAL
  - DESX (used by IPSec)
  - RC5
  - Snuffle :-)
  - streams: WAKE ...
  Actually, there are only a few of them left, and even the ones mentioned above have variable records.... But, there is no new edition of Schneier in the pipeline, so there is plenty of time.

- Alternative PK algorithms and packet formats: DH, ...

- Open PGP (based on pgp5.0) as it is being standardised by the open-pgp IETF group (see page for mailgroup). A working draft for the standard is in progress. Here is a page with all the Open PGP Implementors.

- Examples, documentation (CAN) for the use of Cryptix with Applets.

- PKCS certificates.

- Electronic Commerce protocols:
  - SET
  - ecash
  - SOX
  Or, do electronic commerce protocols belong elsewhere?

Opinions may differ.

### Next Major Release

See the mirror site for the last major release.

The major release will be 3.0.4. It might include:

- snapshot of PGP
- native

## Distribution

These are working notes. I have produced about 4 distributions this way, so I'm getting closer to the truth. We are still prototyping though.

### Introduction

#### How CVS wants to see it

CVS works on the basis of tags which are an advancement on other source code control system that rely on simply dates and version or release numbers (CVS includes this as well).

Tags are alphanumeric labels for a bunch of source files. A tag can only refer to a single version of a source file at once, so when you specify a tag, you get a distribution of the source according to how the tag is currently set. There are several sorts of tags:

- The import vendor tag is meant to describe who the source came from. It is only set during an import, so if you are doing an import, then you are the vendor. You should use your normal user name as the vendor tag (stick to lowercase)

- The import release tag varies with each different release. This is your release number, you set it, so it might increment each time you import a new version. Note that this is probably numerical, and of the form 1-2-3, which refers to your release 1.2.3.

- Distribution tags are created when we export source to users. In this case the team is the vendor (but the vendor tag as mentioned above is only relevant for the importing end-user).

  Distribution tags are used for exporting or getting source. They can move around. Here are some working distribution tags:

  - HEAD is the default you receive when you do a new get or an export. HEAD is the current best in the repository, and doesn't generally need to be mentioned.
  - Cryptix_1-2-3 to indicate that this is the main team distribution of source called version 1.2.3.
  - RPK_1-0-0 for specific distributions of source to a special customer.

- If you ever want to see what tags are attached to a file that you have out, run cvs status -lv which will show everything that it can think of.

CVS wants us to create a distribution by tagging each file version that is in the distribution. That's what we talk about further below.

#### Goals of the Distribution System

We have a number of primary goals in building a release system:

- Simplicity of new releases. So that people can quickly get in there and run the process. I don't want to do this forever, so my system must be easily passed on to others.

- Minimisation of later support hassles. This means that it is impractical to just zip up the lot and send it to the users. What is called for here is a testing phase within the distribution process that runs all unit tests on the very distribution itself, so that we know that all tests will work for users (modulo new bugs).

- Efficiency of new releases. It should be possible to bash one out quickly, so that we can set up lots of new releases (c.f. Cathedral ). This of course is only useful if each release works.

There's also a number of secondary goals - ones that I might sacrifice just to get the ball rolling:

- Production of documentation (i.e., javadoc). This will very quickly become of more importance, in order to reduce support and get users using the software.

- Allow users to incorporate the distribution and then to make their own custom distributions.

- Reproducibility of results. As we get more professional at this, we will need to rebuild releases from time to time. But, in my experience, this is generally avoidable, and is not really of great benefit unless we are supporting some particular client with special needs.

#### Overview of System

All the above considerations leads me to the following rough architecture:

From CVS itself, only do source releases. Then, within the source release, provide the tools (such as the src/build/*.sh scripts) to build the binary distributions.

In order to do the source distribution, all source that has to go in there has to be tagged with the distribution tag. This is a manual task of some tedium (see `util/dist_tag.sh`) This makes sense in that any distribution requires a lot of difficult questions as to what goes in and what is omitted. Each decision is instantiated by applying the distribution tag to each file, meaning that it is chosen for inclusion.

Nota bene that it is the distribution maintainer's job to do this, not the programmers'. The distribution will have a whole bunch of test scripts and so forth that will bounce if not done properly. When programmers wishes to add something into the distribution, they should coordinate with the maintainer on this.

Then, when files are tagged, do a new

       cvs get -r Cryptix_1-2-3 current

The files in this tree will be sticky to 1.2.3, so the process is one of testing, fixing files, changing the tags on fixed files, and then updating this tree.

A script in the source can produce a source release (see `util/dist_source.sh`) or the whole release (see `util/dist_all.sh`). This latter script produces a source release, unpacks it, and produces the binary from that.

A script in the CVS source should produce a source release, which should itself be capable of producing a binary distribution. This binary should be produced from the source release, and then extracted into a temporary directory. There, run the tests. As problems arise, fix the files, and make a new distribution. As fixes work, they need to be tagged with the distribution tag.

When the entire cycle is error-free, and the set of files (actually, revisions) to distribute is tagged, throw away all the above, and do a new

        cvs export -r Cryptix_1-2-3 current

And then run through the process again. The test results duplicate the previous results (this hasn't been tried to date).

Then copy the source zip and other (binary or docs) zips to the archive, and announce the new distribution.

### Tagging

The first step is to decide on the distribution tag. For sake of argument, we assume here it is an increment on the previous one, and that makes it Cryptix_1-2-3.

Then, in a hierarchy, go through the laborious process of tagging each file and module with that tag. Technically this is done with:

       cvs tag Cryptix_1-2-3 [files...]

It's probably worth doing this at least once manually, just to get the feel of the distribution. Notes:

- Name the files explicitly or get the default of the current directory (and it will be recursive unless you add the `-l` option).

- If you want to start from the contents of the previous release, use the `-r old_tag` option:

             cvs tag  -r Cryptix_1-2-2  Cryptix_1-2-3 [files...]

  where 1.2.2 is the previous, and 1.2.3 the new.

- There is a script that automates most of it:

             util/dist_tag.sh util/man/cvs.man

  where the file is a manifest specifying all files and directories. That manifest needs to be read carefully each time, at least to set the release tag.

- Later on, to retag a new (fixed) version of a file, you will need to add the -F flag to force moving the tag to the current version. I use the following in a .profile:

               push()
               {
                   cvs commit $*
                   cvs tag -F ${CRYPTIX_CURRENT} $*
               }
               export CRYPTIX_CURRENT=Cryptix_3-0-3

  and then for each new file that is changed, a simple `push file.java` will commit and tag it. (`dist_all.sh` will do an update in the build tree.)

- This tagging process can be done in any working directory where you can get access to the files (versions) that you want. It is not likely to be the same as the working directory out of which you build the distribution because in that hierarchy you can only see the already tagged versions. For example, it might be your current programming directory.

### Building the Distribution

An important first point is to decide what is in the distribution. These are decided by the files `util/man/source.man` and `util/man/binary.man`. They should be adjusted to include new source and its unit tests. Then they should be retagged.

Once everything is tagged, extract a new CVS directory:

        mkdir 345
        cd 345
        cvs get -r Cryptix_3-4-5 current

Then, the process is one of checking the contents, running the script:

        cd current
        util/dist_all.sh

checking again, and rerunning the script.

Once the distribution gets to the end, and all tests pass, add the `-export` flag (which is only locally set up, this part will need rewriting) to export the distributions up to the web server.

See the util index for more information on the scripts.

Some points to make:

- `dist_binary.sh`, `buildjar.sh`, `testjar.sh`, have to be in the source distribution. This might change.

- testjar.sh makes scripts called auto/testjar* that can be run to reproduce its test actions.

- The scripts are big and hairy, and are only really suitable for big distributions. For day-to-day compilation, the other scripts in util are probably more suitable to local's needs.

- This is probably a Unix only game. Although the scripts might work on a real shell, I suspect the whole approach will need re-thinking in order to work under Wintel.

- `util/dist_all.sh` builds all three current distributions. It effectively does these three:

      	 util/dist_src.sh
      	 util/dist_bin.sh
      	 util/buildjavadoc.sh

  Each of the above can be done individually from any valid source tree.

- The distribution only does what it is told, and just tagging a file for inclusion in the source release says nothing about the eventual release status.

  To adjust the release, change (and push as above) the manifest files `util/man/*.man` and all of the manifests that are included within it. To get a new module into the entire distribution, these should be checked:

      	 util/man/source.man
      	 util/man/binary.man
      	 util/man/JAR.man

  where is JAR is where the binary release goes.

- For each new release, the current system requires that each manifest version number be upped by one, and that the version number in `util/dist_all.sh` be changed.

### Adding Your Code

To get a new unit added into the distribution, mail the distribution team. asking for the unit to be added in, and include:

- name of unit test(s)
- location of files

That first requirement, the name of the unit test(s) is paramount. With that, it is possible to determine the rest. Without that, the addition will take twice or three times as long, as the test will have to be written or cobbled together. This would generally push the addition down in the priorities.

### Some Other Comments

Build and distribution is a core skill for the team to develop, and it is fair to say that we haven't really done much in this area. To help us, here are some docs that comment on the subject.

- Programmers' Canvas talks about some principles of team productions:
  - Build early and often,
  - Work in private sandboxes,
  - Stay in sync, and
  - Prefer merging over locking.

- The Cathedral and the Bazaar also talks about how Linux manages big builds (needs a list of specific sections)

## Programming Standards

### Security

I suspect this is something of an open ended section.

Baltimore makes an interesting claim:

      Note: Any serious Java implementation of cryptographic functionality
      must include a mechanism for obfuscating sensitive information, such as
      keys, in memory. This will inevitably slow down the operation of the
      toolkit.

David:

This is a hard problem in Java (or any garbage-collected language). Obfuscating the keys isn't enough, because there will still be any number of copies of data derived from the keys in memory. That's especially true for the public key algorithms using java.math.BigInteger, since there is no way to wipe intermediate results; you just have to rely on them being garbage collected.

I remember seeing a hack for Linux posted to a newsgroup (comp.security.unix?) showing how to disable virtual memory for an particular program - the example was PGP 2.6.x. It required installing the program suid root, and patching it to make a kernel call (mlockall(), IIRC) then drop privileges.

Installing the java interpreter suid root is a definite no-no for obvious reasons, but maybe we could provide a native C wrapper for Linux that would call it. The newsgroup discussion implied that it was possible to do a similar thing for Windows NT (but not 95 :-( ).

### Getting the Best out of Cryptix

The benefit of strong crypto is best achieved by a balance of code, documentation and testing, as this allows the best use of limited resources.

            power(Cryptix)  = constant * code * doco * tests

Cryptix is now an internationally-used strong crypto package. Now that many people are using the product, it is important that each distribution not generate work for us by causing users to ask obvious questions, trip up on silly bugs, or grumble about the product because there are bits missing.

There are several aspects to the overall process of producing quality code:

- Write a unit test class. A Unit Test is a simple program that just tests the basic functionality. These are essential for distribution, see below.

- Write great doco. See the Documentation section below.

- Write a more complex thrash test. Where there is some uncertainty about the code, a thrash test can pump lots of data through it and try and trip up problems based on the type of input. It might:

  - encrypt-decrypt-check from a source of random data
  - compare results against a known good source by taking two streams in, processing the first and comparing with the second.
  - work from a seed by doing a basic test, then hashing the seed and repeating.

- Write an example program. This would seek to do something generically useful with the class:

  - encrypt a file
  - generate performance benchmarks
  - examine the data generated by other tools (e.g., pgp keyring revealer).

  Don't forget to include a simple self test so that the distribution can pick up silly errors.

Which of Grandma's eggs you wish to suck is dependent on the moment!

### Unit Tests

Testing is no longer a nicety in Cryptix, it's a necessity. Cryptix has grown to the point where it can no longer be tested manually, by just running a bunch of programs and looking at the results. During distribution, each part of Cryptix is tested (sometimes multiple times) to prove that it is good.

The objective of the distribution testing is to eliminate errors in the distribution phase. It cannot eliminate coding errors, and it can't improve the basic product.

Without automated testing, the complexity and size of the product is such that there will be sufficient errors in every distribution such as to make it unusable. So in a sense, the objective can be read as re-making Cryptix usable for the users.

To meet the objective, the distribution scripts needs to prove that each part of Cryptix is present and working. This is done by running the unit tests which are, literally, tests of component units of the whole.

The main feature of these tests is that they provide a positive indication that the code is present, and that it is working.

A unit test should conform to the following:

- Testing should be sufficient to conclude:

  - Class files are present and available for linking. Especially, watch out for inner classes that are compiled into distinct `parent$inner.class` files.
  - The basic mechanism of the code works.
  - Major integration is working, e.g., cipher modes.
  - The major API calls are working.

- Algorithm tests - those of ciphers, digests and PKs - should

  - Test the basic cycle of operation against a pre-calculated set. That is, it is insufficient to simply encrypt-decrypt and check. A cipher should compare the ciphertext against known good ciphertext.
  - Include all known conformance sets. This allows the documentation to assert that the implementation conforms to the stated standard, without having to be worried about when and which version was tested for this.

- Testing is called by identifying the entry class. This is normally the main of a custom test class, for example `cryptix.test.TestLOKI91`, but it doesn't have to be. If another class is used, it cannot, however, be inaccessible (i.e., it cannot be private).

- The unit test is run without arguments. This is more a saving for the distribution scripts that cannot remember which arguments are relevant. The same program can still be used, with arguments, to achieve more comprehensive tests.

- Positive indication is returned, being one of the exit codes in Table 2, below.

- The unit test should take less than a minute, and hopefully less than 10 seconds. This is because the distribution process runs dozens of tests, and if each are long then the overall process becomes too long to debug.

- The unit should not be circularly dependent on any other, and especially, neither should a unit test rely on modules that are not yet tested themselves. Hierarchical dependencies are a natural part of good building block design, but when blocks start to prop each other up, problems arise.

  ( Explanatory note. The main example here is the current IJCE tests. The test that is run, cryptix.test.TestIJCE is also dependent on the Cryptix Provider being present. This means that basic failures in the IJCE are not picked up immediately, and that results returned by this test must be further dissected into the units involved. )

Once the unit test is ready, send the class name entry point to the distribution team.

In fact, sending in the test is the most important thing that should be done, if the issue is getting the code into the distribution. From the test, it is fairly easy to work out what code is required. Without the test, the distributor needs to look for the test, which is fraught with errors, or to write a test, which is inefficient.

#### Native Testing

This has not yet been tried at the distribution level. The testing of native code currently borrows from the existing algorithm unit tests, which leaves a number of shortfalls as implied below.

Unit testing of native code will need to ensure that

- both implementations are tested
- returns a positive indication that both are working
- returns a failure otherwise (perhaps returning the exit code 4, carry on, if only one of the two is working)
- can distinguish between "native is supposed to be there but doesn't work" and "native is not written and should not be there."

One possibility is to run each unit test twice, once for native, once for java. This could be done simply by including different entry points into the tests:

        java cryptix.test.TestLOKI91.native
        java cryptix.test.TestLOKI91.java

Another possibility is that the unit tests could be upgraded to cope.

This is not a big issue in advance, the most important step is to get running native code on the distribution platforms (Unix) so that the strategy can be worked out.

#### Exit Codes

A test should exit according to the conventions in Table 2, which are interpreted by distribution scripts.

| no failures | failures
no successes | no tests = 3 | bad = 1
successes | good = 0 | carry on = 4

Table 2: Matrix of Test Exit Codes

The purpose of the mixed-results exit code 4 is to allow the distribution scripts to carry on and test the entire set of tests, without forcing a debugging phase for each single bug found. It's a problem of efficiency, as normally found in compilers' reporting of errors and so forth.

If the carry on result cannot be provided, then bad indication of 1 should be returned.

In Java, to return the code, call `System.exit(n)` where `n` is the exit code desired.

### Documentation

Any documentation and testing that you can supply will be immensely appreciated by the users.

Documentation can defer questions being asked over and over again on the mail groups. By making the job of the user easier, we make the amount of support less and can spend more time on new code.

There are several ways you can do this:

- Write comments in the code that describe what's going on (not what the code means, it can be assumed that the reader well-understands the language).

- Write a javadoc section at the top.

  The Javadoc is the place for any references to standards or papers from which the code is derived, and the source of any test data for conformance purposes.

- Tests make for good documentation, as discussed above. For some people, tests are the only documentation, after that it is time to read the source.

- Write a Cryptix Application Note to describe some deeper use of the code. This is intended to explain or document something that cannot be done in the code.

Everybody is encouraged to work together on updating the documentation; please be careful about preserving other people's literary achievements.

### Applets

There are problems with some browsers (I.E. and HotJava) if using Cryptix from applets. Some of the examples allow arbitrary files to be read or written. To combat this:

- Examples should not be included in the straight classes jar files. They can happily go in a separate jar file.
- Application classes should be non-public.
- Application classes wherever possible should use System.in and System.out for I/O (i.e. act as a filter) rather than using FileInputStream, etc.

Historical note: there may be more problems using Cryptix from applets, as Cryptix was never intended to be used by applets.

### Whitespace

Our working white space convention is:

- 4 spaces should be used for each level of nesting,

- comments on the left using spaces,

- Tabs should not be used at all, as they suffer mangling when they go through the various conversions.

- Line endings in Unix format (LF only).

In the source release there is a Java application called `util.FixLineEndings` that can be used to convert tabs to spaces, and line endings to any format. Make sure that the root of the source release is on the CLASSPATH, then do "`java util.FixLineEndings`" to get a summary of options.

There are also shell scripts to convert the whole source tree at once. For example:

> `sh util/unixify.sh`
> or
> `sh util/dosify.sh`

CVS should probably deal with CRs. Iang's guess at this stage (not having tried it) is to strip out CRs on input, and add them in on output (only for Windows users). This is a known problem for which there are known fixes.

### Final Keyword

Fields should be final iff they are constants.

Classes should usually be non-final, unless they need to be final for security (example: cryptix.provider.cipher.*).

Methods should usually be non-final, unless

- they need to be final for security (this is quite rare).
- they are implementing an existing API written by someone else that was declared as final (example: some of java.security.*)
- they are auto-generated (example: the Maker classes generated by JavaCC)

      > It speeds things up, and may prevent security holes where
      > people extend our classes deviously, but is it appropriate
      > in a library?

Static and private methods are called using static binding, so they are just as efficient as final methods (the 'final' qualifier is redundant for these methods, although there are quite a few cases in Cryptix where it is there anyway). In the cipher and md classes, almost all of the methods that need to be called quickly are already private.

### File Name Conventions

- Stick to a 32 character maximum for filenames for now (and remember that ".class" takes up 6 of those characters). Many less popular platforms are still plagued by this limit. Watch out for inner classes that produce longer compiled class names.

## Politics, Culture, etc

This entire section is rantings by iang who is probably not of the same opinion as anyone else...

### How is the team organised?

Organised? Not very.

In political terms, we are a socialist taskocracy or somesuch. That is, power is shared amongst those who do the work. Hacker power rules.

This effectively means that those who make the distributions say what goes in them. Those who write a particular interface have final say as to how it is done.

Those who write original code get to say how it is done. By doing it. But they don't get any rights in later versions, as later editors have the same rights.

The counterbalancing to this tremendous concentration of power in the hands of the do-ers is the ability of the rest of the world to ignore, rewrite or fix anything they do not like. Ignorance is the privilege of the market, and a message for the developer; rewriting or fixing converts critics into do-ers.

All this leads to several effects:

- If you do not like the way things are going, do it your own way. Including, making your own distribution.
- The onus is on the original developer to document and explain what was intended, so that gratuitous hacks by well-meaning disciples are limited to the bits that really do need fixing.

There is an well-written paper by Eric S. Raymond called The Cathedral and the Bazaar that more or less agrees with this viewpoint. (However, it more or less disagrees with the following "by invitation" approach).

Another description is the chapter on "The Rise of ``Worse is Better''" in an article on Lisp history by Richard P. Gabriel. But this probably goes in a general software engineering section.

### Why is the team "by invitation?"

Here is a mostly unedited mail describing the viewpoint of iang :

I think there are a lot of issues, which add up in my experience to a conclusion that I think the group will work better if it is managed as an "invite-only" club.

I am not sure what all these things are, but I'll have a go at it:

Industrial Espionage: yes, that's a concern. Whilst pure competition doesn't count, there is plenty that an aggressive producer of crypto could do to the group to kill it if the name of the game was to cause Cryptix to wither and die. If I was Baltimore, I could have a lot of fun. Even if I was Sun, I would have a lot of ability to influence the fate of the team if I could see everything that was going on in it.

Team Size: there is an issue here in managing the amount of activity. Whilst it is good to have lots of code happening, there is lots of support that needs doing. At the moment, things are quiet, but over the period of the release, many man-weeks were consumed in support activities (femme-weeks can be included at premium rate here :))

If the team gets much more productive (that is, releases start happening more quickly), then there is going to be a need to boost up all the support side. That means things like the CVS and automatic distribution scripts. Now, these are not technically out of our reach, but someone has to do all the work. We have talked about it in the past, but it is no exaggeration to say that the non-coding aspects of Cryptix can be extremely costly as well as boring.

Noise level: with a team of this nature, it is important to keep everything focused. If we have non-coders, or inactive people, that are not focused on writing code, then there is a temptation for them to get involved and waste other people's time. It's bad enough when we disagree amongst ourselves, but at least we are all relatively quick to work out when the code is suffering and make some sort of judgement as to whether a discussion is still valuable.

So my current viewpoint is to keep my eye out for people that are producing hot crypto and to try and make sure that they are willing active contributors. For e.g., there was a company that wanted to join, but as far as I could see they were going to keep their efforts proprietary and just incorporate the Cryptix bits were there were gaps.

This is a working policy, not dogmatic truth. It's a combination of logic, experience and paranoia. So, comments eagerly sought.

In particular, I am thinking about The Cathedral and the Bazaar which says we should open up.

### How can I help someone into the team?

If you know of a person that wishes to "get involved" then try this strategy:

- Pick an area that is smallish and not really involved such as a secret key algorithm or a new packet format. Start the person on that, and be their interface to the team. This way, they can concentrate on the code.
- When the code is done, submit it into the archives in the normal way.
- If your friend is still willing to continue, then sponsor the person into the team, and help with the selection of further tasks.

### Culture

Nope, no culture yet. Give it another thousand years and some yeast and hops.

### A Rant on the License

Follows is another slightly dynamic rant by iang:

            The goal of the Cryptix license is to efficiently
            allow freeware crypto to be published.

The license does not change the ownership of the code, that remains vested in the original owner. So, if your University applies the rule that all student works are owned by the University (as many do), then this is no problem as the University can be stated as such: the owner of the code. Although we might want to better reflect that distinction in the FAQ, which is where it is currently preferred to list such issues, once, in full.

The Cryptix license also has no effect on the authorship of the code. The current method is to put the author in the @author tag, as code is written and changed.

What the license does do is assert copyright on behalf of the owners (not the authors, I guess), and it names them as the Cryptix Development Team. This then becomes a device for protecting the code against attacks. It is also generic and common, which means that we save a lot of hassle in distribution time: one notice throughout the system means ease of packaging and ease of understanding by readers.

The motive for using Systemics as the asserter or claimor of the rights (on behalf of...) has changed over time. Originally Systemics wrote the code, so it was obvious.

Now, Systemics doesn't write (much of) the code, and doesn't wish to (much). But there is some value to the group in maintaining a corporate umbrella over the product. Copyright is normally asserted by corporations and not individuals for the simple reason of limited liability. That is, if one of the PGP packets goes astray on the way to NASA's latest launch and the bright new space station decides its geosync orbit is just north of the Empire State building, then some questions are going to be asked...

In this case, Systemics makes a useful protective device for the individuals. Basically, when the squillion-dollar lawsuit lands, Systemics goes under, all assets sunk, and we all move onto fat talking engagements about how we sunk the space station. I am somewhat flippant about this because a. it's unlikely, and b. it's a risk of business, and c. I have dealing with companies of this nature for years. It's also worth noting that companies and lawyers know what is going on. The game is familiar to all.

There are other considerations.

Of course this comes at a cost. I gather the DebIan (sp?) crowd of Linux hackers have recently incorporated (I suppose for these reasons) and it has cost them a couple of thousand dollars. Which they don't have, donations kindly accepted :))

There is also a risk to Systemics that the boys at no such agency will phone up the regulators and put the squeeze on. Fine, Cryptix ends up squeezed all over the net, and Systemics a. folds, or b. disowns it.

The current benefits to Systemics are of course some useful kudos, and some possibility of picking up consulting work. As far as collected revenues go they have not been worth squabbling over so far :-/ but the future holds more potential. Indeed, with recent activity, there is a plausible upside for Systemics, as there is for the team.

However, it is an open market. Any member of the team is free to sell software or consulting or work based on Cryptix; the license ensures that everyone has at least an equal chance of doing well. So Systemics has to maintain a fair balance, else risk breaking up the team.

## Team-FAQ History

> $Log: FAQ.html,v $ Revision 1.13 1999/07/26 15:46:08 iang elimination of old systemics.com domains, emails, etc. Revision 1.12 1999/03/05 19:18:40 gelderen
> - added putty. Revision 1.11 1999/03/02 20:55:13 gelderen
> - added SSH comment Revision 1.10 1998/06/01 19:33:53 iang
>   - updated the unix ssh cvs access notes
>   Revision 1.9 1998/04/26 18:58:19 iang
>   - formatting of revisions (again).
>   Revision 1.8 1998/04/26 18:56:20 iang
>   - Oops, previous was correct, the cryptx repository is odd, not the FAQ.
>   Revision 1.7 1998/04/26 11:39:21 iang
>   - dropped cvsroot from home in ssh line for cvs access. wrong.
>   Revision 1.6 1998/02/17 20:55:05 iang
>   - correction to Edwin's name
>   Revision 1.5 1998/02/15 17:56:32 iang
>   - fixed silly comment
>   - actually, the new members were added to the project list.
>   Revision 1.4 1998/02/15 17:53:58 iang
>   - zox address wrong
>   Revision 1.3 1998/02/15 17:52:43 iang
>   - added updates
>   Revision 1.2 1998/01/31 23:38:31 iang
>   - moved from ../cryptix/team-FAQ.html
>   - not at later version, needs to jump to 1.15 or later
>   Revision 1.11 1998/01/12 21:50:52 hopwood
>   - Moved history to end of file.
>   - Fixed some typos, cosmetics.
>   Revision 1.10 1998/01/11 23:59:00 iang
>   - added section on how to add new units.
>   - reformatted history.
>   Revision 1.9 1998/01/11 17:01:59 iang
>   - Added substantial section on unit tests.
>   - Reviewed build.
>   Revision 1.8 1998/01/09 22:34:34 iang
>   - added section on 32 character filenames
>   Revision 1.7 1998/01/06 20:14:24 iang
>   - team-FAQ: updated to review distribution current practice
>   - Support: a few updates from post
>   - Compilation: ?
>   Revision 1.6 1998/01/02 21:43:52 iang
>   - updated current projects
>   Revision 1.5 1998/01/02 21:31:16 iang
>   - added reference to NIST quest
>   Revision 1.4 1997/12/30 01:26:19 iang
>   - added refs for maths algorithms in HB/AC Menezes to wish list
>   Revision 1.3 1997/12/23 19:20:17 iang
>   - press release finished off, as published, mostly
>   Revision 1.2 1997/12/23 15:31:41 iang
>   - Copied in my working team FAQ to this location, which is at least visible.
>   - Many Many Changes, this one was well out of date.

Copyright © 1997 Systemics Ltd
on behalf of the Cryptix Development Team.
All rights reserved.
Cryptix is a trademark of Systemics Ltd.
`