LISTSERV FILEDOC Oct 92 LISTSERV File Server Functions INTRODUCTION This document is an introductory guide to the file-server functions of LISTSERV that are available to general users. Throughout the document, we will use the term LISTSERV to refer both to the Revised LISTSERV software (also known as Eric Thomas' LISTSERV) and to LISTEARN (derived from Revised LISTSERV and maintained by the EARN Association). A separate chapter is provided at the end of the document which highlights the difference in commands, parameters and functions between the two products with regard to this particular service. AVAILABILITY All active LISTSERV servers have file server capabilities. Please refer to your local management to locate the server nearest to you. AVAILABILITY OF SERVICE SOFTWARE LISTEARN is available from the EARN association for all EARN members. To request a copy of the code or for more information write to TURGUT@FRORS12.BITNET. Revised LISTSERV is available for all the sites connected worldwide to the NJE network, excluding the EEC countries, from Eric Thomas (ERIC@SEARN.BITNET). Both products are available free of charge, but subject to the signing of a licence contract. Please contact the addresses above for details. ACCESS The file server functions of LISTSERV are available to any user on any electronic mail network world-wide as long as certain conditions are met: - the user is able to send an electronic mail message to LISTSERV - the message complies with the electronic mail standard RFC822 - LISTSERV can construct a valid return address from the message An NJE user (user of a computer directly connected to EARN/BITNET or any of the cooperating networks) can access the file server functions by: - interactive message - electronic-mail - file transfer To send a command to LISTSERV by e-mail or file, simply write the command and the necessary parameters on the first line of the mail body or of the file and then send the mail/file to LISTSERV. Any line appearing into the mail body or file will be interpreted by the server as a command and will cause either the execution of the command or a diagnostic message to the sender in the case of a syntax error or unrecognized command. DESCRIPTION Although the primary function of LISTSERV is to distribute mail and files according to predefined distribution lists, a few basic file server functions providing powerful list-based file access control and remote file updating facilities, under the control of both the list owner and the LISTSERV management, are also available. The main purpose of these functions is to provide the subscribers of a list with a set of data or program files which are related to the list. Apart from the obvious example of list "notebooks" (archives of messages which appeared on the list), working groups can provide minutes of internal meetings held by some of the subscribers, technical groups can share application programs related to some software they are all using, etc. Automatic distribution of updated materials is also available to subscribers. Last but not least, a set of public files open to all and not associated with any particular distribution list, can be made available. Files can also be maintained individually by "file owners" (with storing privileges, similarly to the "list owners" but only related to specific files). Note: In order to prevent abuse of the file server functions, and thus causing excessive load on the network, there is a maximum amount of information which may be retrieved by a user in a day. Each LISTSERV site can determine its own maximum. The default is 256k per day. LISTSERV file commands conform, more or less, to the standard command set of the NETSERV file-server developed by Berthold Pasch for EARN. Please note that a significant number of the commands have the same syntax for LISTSERV and NETSERV and yet may operate very differently on the two servers (the best example being the PW command). Please refer to the NETSERV documentation if you want a detailed description of the NETSERV commands. The filelist structure - In order to allow each list to have its own set of files, independent of other lists, and to make it possible to hide the very existence of private files from those users who are not supposed to retrieve them, LISTSERV keeps listings of available files in special files called FILELISTs. A FILELIST is like a directory which can contain subdirectories (other FILELISTs). Any level of nesting is allowed. The "root" filelist is called LISTSERV FILELIST and is always maintained by the LISTSERV management at that site. LISTSERV filelists are almost fully compatible with the ones you may obtain from NETSERV. The header of the filelist will provide information about the contents of the filelist directory and a list of File Access Codes (FACs) which will be used later in the body of the filelist to identify who is authorized to retrieve the files or store them onto the server. This FAC list is included between "banners" made of an asterisk, a blank and a full line of colons, and will usually have been commented by the maintainer of the filelist to provide more information as to whom the FACs refer to. The body of the filelist will contain file declarations, possibly interspersed with comment lines. Comment lines start with an asterisk in column one. The format of the file-declaration lines is as follows: +----------------------------------------------------------------------+ | | | fname ftype get put rfm lrecl nrecs date time description | | | | | | | | | | | | | | SAMPLE FILE ALL CTL VP 106 85 92/06/22 19:50:14 Dummy file | | | +----------------------------------------------------------------------+ where: FNAME is the filename FTYPE is the filetype GET is the FAC that controls read access to the file, that is, people who are not included in the FAC will not be able to get the file. PUT is the FAC that controls write access to the file. As a rule, it points to the file maintainers. RFM is the record-format of the file. The first letter is either "V" for "Variable length records" or "F" for "Fixed length records", while the second letter will be set to "P" for "Packed format files" or left blank for non-packed files. Note that packed files will be automatically unpacked prior to being sent to you as a result of a GET command or Automatic File Distribution process; however, if you set up a direct link to the disk on which they are stored (eg, by FTP), you will find them to be in packed format. NRECS is the number of records in the file. Files flagged with nrecs = 0 and dots in the other fields are not yet available but can be subscribed to or stored by their maintainer. DATE/TIME is the date the file was last updated, in yy/mm/dd format (this makes it easier to sort the filelist by date). Note that the process of updating the "date" field in a filelist causes the filelist itself to be flagged as changed and the corresponding date/time in its entry up the tree structure to be modified accordingly. The Packages concept The term "package" refers to a set of program or data files which are supposed to be retrieved together under normal conditions. Provision has been made to allow users to retrieve these files with a single command, and to subscribe to the package as a whole with updated versions of only the individual changed files being sent out. The package exists independently of its individual components which can still be referenced separately. There is a dummy file and a definition file associated with each package. The dummy file has a fileid of "package-name PACKAGE" and does not really exist. However, ordering this dummy file will cause the whole set of files to be sent to you. Similarly, subscribing to this single dummy file will result in a global subscription for all the components of the package. As a rule of thumb, the dummy file should be used whenever reference is made to the package as a whole. The definition file has a fileid of "package-name $PACKAGE" and lists the set of files that are contained in the package. Unlike the dummy file, it really exists and can be retrieved as any other normal file. It will usually list itself as the first file of the package so that you get a copy of it and check that you have received all required files before starting to use the package. It may reference another package which is required for the package you ordered to be used (any level of nesting is allowed). If you subscribe to a package via Automatic File Distribution (qqv), you will automatically receive update versions ofd all the corresponding sub-packages if they are updated. Similarly, ordering the package will cause all sub-packages to be sent to you. Notebooks LISTSERV is able to automatically keep notebooks for a list if the list owner so desires. Those notebooks are listed in a special filelist called "NOTEBOOK FILELIST", which is automatically maintained by the server according to the parameters specified in the headers of its various lists. The files of the notebook will also be automatically appended to the corresponding "listname FILELIST" (if it exists) to make it possible to get a listing of all archive files kept for a particular list. If no "listname FILELIST" has been prepared by the list owner, LISTSERV will automatically create a dummy one with no file-definition entries other than the ones for the notebook files kept for the list. Retrieval of the NOTEBOOK FILELIST has been restricted to registered Node Administrators (NADs) as it is usually a very large file prone to generate significant network traffic. The files listed in the NOTEBOOK filelist are just like normal files and can be retrieved as indicated by their GET File Access Code. They can be subscribed to as normal files, and can even be modified or deleted by their owners, as indicated by their PUT FAC (which usually points to the list owners). However, once such a file has been deleted, it will disappear permanently from the filelist, unlike a normal file which would be forced to nrecs = 0 but would remain listed. Any archive file can be ordered either as "filename filetype listname" or as "filename filetype NOTEBOOK". The two syntaxes will produce strictly equivalent results, regardless of whether the NOTEBOOK filelist has been made available to general users by the local LISTSERV management or not. However, it is recommended that the latter form be preferred in application programs as it will result in better response time from the server: the NOTEBOOK filelist always denotes list archive files whereas the listname filelist might contain other files, which makes it impossible to optimize the search algorithm. COMMANDS INDex Use the INDex command to review the contents of a filelist as follows: +----------------------------------------------------------------------+ | | | INDex | | | +----------------------------------------------------------------------+ where: FILELIST_NAME is the name of the filelist you want to search FFORMAT is the format you wish to receive the document. The default format should be acceptable, except under unusual circumstances, so this parameter is rarely needed. FCLASS applies only if you can receive the document as a file (NJE users) and specifies which VM/CP spool class shall be used to send you the document. It can be one-character alphameric (A through Z, or 0 through 9). Please refer to the IBM/CP Command Reference manual for more information. The server sends you the specified filelist (defaults to LISTSERV FILELIST). This command is strictly equivalent to "GET filelist_name FILELIST" and has been made available for compatibility with other file servers on the network. GET Use the GET command to retrieve a file (or a filelist as with: INDEX filelist-name) +----------------------------------------------------------------------+ | | | GET fn ft | | | +----------------------------------------------------------------------+ FN is the file name FT is the file type (the "extension" of the file) FILELIST_NAME is the name of the filelist under which the file is stored FFORMAT is the format in which you wish to receive the document. It can be: Netdata, Card, Disk, Punch, LPunch, XXEncode or UUEncode. If you are at an NJE node (i.e. in the EARN/Bitnet network) the default will be the value that has been defined for your node by the system administrator of your node. If more than one has been defined the most efficient one is chosen. If you are not an NJE user (e.g. you write to LISTSERV from an Internet node) the result will be sent back to you as a mail item. The only options that apply, in this case, are UUEncode or XXEncode, for the standard UU, or XX, encoding of the file. This is useful for getting non-text files. Note that you will have to decode the file yourself afterwards. FCLASS same as in the INdex command. The server sends you the requested file provided you are authorized to retrieve it. You can use the GET command, or its synonym SENDme, to request a file to be sent to you. You must specify the filename and filetype of the file as they appear in the filelist, and you may optionally append a third parameter to the command to indicate from which filelist you want the search for the file to start. This parameter is normally not required unless there is more than one file available from LISTSERV with the filename and filetype you are requesting. Application programs which issue GET commands to LISTSERV and know the filelist name should specify it since it speeds up the filelist search. The default is to start at the root filelist, LISTSERV FILELIST. GETALL The GETALL command can be used by filelist owners to get files subject to access restrictions. +----------------------------------------------------------------------+ | | | GETALL fn ft | | | +----------------------------------------------------------------------+ FN is the file name FT is the file type (the "extension" of the file) FILELIST is the name of the filelist under which the file is stored FFORMAT same as in the GET command. FCLASS same as in the INdex command. The server sends you the requested file provided that you are either authorized to GET it (refer to the description of the GET command for more details), or you are authorized to PUT the filelist that defines it or any other filelist in the same branch up the tree structure. The GETALL command allows filelist owners to retrieve files defined in a filelist that they can PUT, but which they are not allowed to access via the GET command because their File Access Codes prevent it. That is, whenever the command originator would be able to change the File Access Code that prevents him from retrieving the file by modifying the filelist in which it is defined, he is allowed to obtain the file directly by means of a GETALL command. The GETALL command will only bypass the File Access Code access restriction. If any File Access Verification Exit has been established for the file, it will still receive control and might still deny access to the file. The LISTSERV operation staff normally has PUT access over the root filelist, LISTSERV FILELIST. This implies having GETALL access over all the files stored on the local server. GIVE Use the GIVE command to have the server sending a file to any user in the network. +----------------------------------------------------------------------+ | | | GIVE fn ft userid@node | | | +----------------------------------------------------------------------+ FN is the file name FT is the file type (the "extension" of the file) FILELIST is the name of the filelist under which the file is stored USERID@NODE is the full address of the user you want to receive the file FFORMAT same as in the GET command. FCLASS same as in the INdex command. The server sends the requested file to the specified userid, provided you would be authorized to retrieve it yourself by means of a GET command. The destination userid will receive a mail message saying that you requested the file to be sent. You will get a copy of all messages sent to destination userid so that you are informed if some error occurs while sending the file. The GIVE command allows you to ask LISTSERV to send a file or package to another network user. This is very useful when you want to send a file to some user who is not authorized to retrieve it but who is topologically nearer to the server than to you. It also allows you to order files for people who are not able to issue the requests themselves, for example people behind a one-way gateway which allows mail to be received but not sent out to LISTSERV. For technical reasons, it is not possible to GIVE a LIST file: the present inter-server command protocol does not allow a destination address different from the command origin address, which would make it impossible to consistently forward the GIVE request to other servers. LISTSERV vs LISTEARN additional features The only different feature for the file server functions are related to the values that can be specified for the fformat option for the commands described above. Additional values for the fformat option specific to LISTEARN: BTOA uses the UNIX encoding (binary to ASCII) conversion. HEX turns each byte in the file into two bytes. EBCDIC turns an ASCII file into EBCDIC. ARCHIVE, VMARCHIVE, VARC compresses the file in VM-ARC format. SPLIT is used to split the output file into small files. It can also be used in addition to any of those above, separated by a comma. (e.g. GET myfile log90 f=XXE,split) Additional values for the fformat option specific to Revised LISTSERV: VMSDUMP useful for getting files on VAX/VMS. MAIL useful for getting a response by mail, when sending requests by interactive message. MIME/TEXT converts text files according to the MIME standard. MIME/APPL converts non-text files according to the MIME standard. CLIENTS No specific clients are available for the use of this fileserver feature. ONLINE INFORMATION This document is designed to be used in conjunction with the LISTSERV User's Guide: LISTSERV MEMO and assumes basic knowledge of the LISTSERV functions available to general users. File owners should refer to the: LISTSERV File Maintainer's Guide: LISTFOWN MEMO (available from LISTEARN only) for a more technical discussion of the internal format of filelist entries and a description of the privileged file-maintenance commands available to file owners. OTHER SOURCES OF INFORMATION Title: Setting up the LISTSERV File Server Author: Benjamin E. Chi Coverage: files Date: 11.7.91 Source: LISTSERV@ALBNYVM1: FSV GUIDE