LARKEN ELECTRONICS

SEQUENTIAL / RANDOM ACCESS

FILE UTILITY

By
Larry Kenny

Edited and Annotated By
David Solly

HTMLized and Edited By
William McBrine


TABLE OF CONTENTS


Forward to this Edition

This instruction manual is derived from the instruction pamphlet provided for the Sequential Random Access File Utility by Larry Kenny when the program was originally issued in 1988.

Most of the editing was done for the sake of saving the reader's eye-sight and for easier transfer to electronic media such as this web site. Otherwise, this instruction manual is as Larry Kenny wrote it.

David Solly
Bibliotheca Sagittarii
Ottawa, Ontario
March 24, 2000

Copyright

This is copyrighted material. It is provided for users of the original software who require a manual, or for personal reference. For reproduction for any other purpose, please write for permission to the following e-mail address: ac355@ncf.ca (David Solly)

Bibliographic Information

Kenny, Larry

Sequential random/access file utility / by Larry Kenny. – Edited and annotated by David Solly. – Ottawa, ON : Bibliotheca Sagittarii, ©2000.

– "Larken Electronics, Navan ©1988"-t.p.

– Includes index.

Conventions

Punctuation

  1. Guillemets (« ») are used to replace standard quotation marks (" ") where they are required for grammar. This is to distinguish quotation marks being used as grammatical boundaries from the same marks that are required in a program or command line.
  2. Brace brackets ([ ]) are used where there are optional parameters in a program or command line.
  3. This font has been used throughout for lines of programming.
  4. Italics are used to indicate computer responses printed on the screen.

Key Words

  1. A "disk" usually means a double-sided, double-density 5¼ inch floppy diskette. A few Timex/Sinclair users did have 5¼ inch quad-density drives at that time and 3½ inch drives were just becoming popular.

General Introduction to Channels

The Sequential / Random Access File Utility (henceforth: LKS/R) is a RAM-resident extension to the EPROM-based Larken Disk Operating System. It allows LKDOS-2068 to handle large database files and exploits a number of new features of the Timex/Sinclair 2068 home color computer (henceforth: T/S 2068) that have been previously unused. Up to four files can be accessed at once, to allow data to be sorted or merged.

Built into every T/S 2068 is the capability to use up to fifteen devices for data input and output. The different devices are connected, in software, through the T/S 2068's channels. The main devices used for the T/S 2068 output are the upper screen, lower screen and the printer. The main input device is the keyboard. The commands PRINT, INPUT and INKEY$ are used to send and retrieve data from these or any other devices. The commands in their plain form default to the commonly used channels.

Example:

Command Default Channel Description
PRINT #2 Send output to the upper screen.
LPRINT #3 Sent output to the T/S 2040 printer.
INPUT #0 Input data from the keyboard. Print to the lower part of the screen (Ch#0).
INKEY$ #0 Read from the keyboard.

All these commands can be directed to other channels by using a channel identifier after the command.

Examples:

10 PRINT #0; "HELLO"

This line will print the word "HELLO" on the lower part of the screen.

10 PRINT #3; "HELLO"

This line will print the word "HELLO" on the printer.

INPUT # and INKEY$ # can also be used to read data from channels other than the keyboard (Ch0).

The LKS/R utility allows you to send data or read data from the disk drive using PRINT #, INPUT #, or INKEY$ #. The command syntax for PRINT #, INPUT #, and INKEY$ # is the same as for INPUT, PRINT, and INKEY$.

Loading the LKS/R Utility

Insert the supplied disk into the drive then type: RANDOMIZE USR 100: LOAD "SRcode.B1". This loads the operating code in as a large REM statement. Type RUN to initialise the BASIC memory map. (This moves the start-of-BASIC pointer up past the REM statement so it becomes invisible.)

The OPEN # Command

Before you can use these commands to access a file on disk, you must first use the OPEN # command to create a file on the disk or re-open an existing file. The OPEN # command also puts a pointer to the channel in the T/S 2068 stream table so that the PRINT # and INPUT # commands see the channel as open.

Note:

The LKS/R uses the same command structure as LKDOS. LKS/R commands are preceded by a RANDOMIZE USR 26800. The LKS/R is resident in RAM just below the memory location PROG. Don't confuse the PRINT # and INPUT # commands with the «PRINT #4:» command prefix used for LKDOS or Larken Extended BASIC.

The PRINT #, INPUT # and INKEY$ # commands which are used to send and read data from a file are not preceded by the LKDOS «RANDOMIZE USR» prefix.

The LKS/R uses the OPEN # command in two modes: output mode and random access mode. The syntax is:

RANDOMIZE USR 26800: OPEN# channel, "file name", OUT : REM Output type

RANDOMIZE USR 26800: OPEN# channel, "file name", RND : REM Random type

(OUT and RND are tokens on the «O» and «T» keys. channel is a channel number from 2 to 15)

Output Type

The output type is used to create a new file or to add more data to an existing file. It can only be PRINTed to and can not be INPUT from. Trying to read from an output type file will result in an error message. If the file already exists, then the file data pointer (internal in LKS/R) will be set to add new data to the end of the existing data.

Random Type

The random type is used for existing files to PRINT or INPUT (or INKEY$) data to or from. Data can be read sequentially from the start of the file to the end; or, using PRINT #channel; TAB record number; command, data can be randomly accessed from anywhere in the file, and read or changed. In random mode, the file can not be lengthened. When a file is opened as a RND type, the file data pointer will be set to the start of the file so that the first data accessed will be the first in the file.

CLOSE# COMMAND

After you're done accessing data from a file, you must close the file with the RANDOMIZE USR 26800: CLOSE# channel command. This ensures that all data is sent to the disk, and disconnects the stream from the channel so data can't be read or sent.

Here's an example program that opens a file as output, then sends data to the file, then closes the file:

10 REM Create file program
20 RANDOMIZE USR 26800: OPEN# 7, "TEST",OUT
30 FOR A = 1 TO 10
40 INPUT #7; "THIS IS RECORD # "; A
50 NEXT A
60 RANDOMIZE USR 26800: CLOSE# 7

Running this program will create a file on the disk. To see the contents of this file type the following LKDOS command:

RANDOMIZE USR 100: PRINT "TEST".

The next example program will read the file back a record at a time using the INPUT # command. The file is opened as a random type:

10 REM Read file records
20 RANDOMIZE USR 26800: OPEN# 7,"TEST",RND
30 INPUT #7; A$;
40 PAUSE 60: REM Delay to show records are read one at a time
50 PRINT A$
60 GOTO 30

You will have noticed that the T/S 2068 makes a noise when using the INPUT # command. This is just a small bug in the T/S 2068 ROM left over from the key scan routine.

An End of File error is produced when a file is read right to the very end. Even though the file is at the end, it's still open. You can move the file pointer to the start of the file by using the command PRINT #7; TAB 0; (described later on), or you can close the file and re-open it.

Warning!
An open file (output or random) must be closed before you remove the disk from the drive. Failing to do so will corrupt the data on the disk.

Here is another example of reading a file — this time using the INKEY$ # command. INKEY$ reads a character at a time and does not produce the key click noise.

Warning!
Be sure to close the file from the previous program with RANDOMIZE USR 26800: CLOSE# 7 before going further.

10 REM Read file using INKEY$
20 RANDOMIZE USR: 26800: OPEN# 7, "TEST",RND
30 LET A$ = INKEY$ #7
50 GOTO 30

INKEY$ #

INKEY$ # is useful if you're searching for a specific character. It can also read characters, such as non-ASCII or control characters, that cause INPUT problems.

After reading the last character of a file, the LKS/R will return a CHR$(266) (i.e. STOP) as its next character before causing an End of File error. Using INKEY$ #, it's possible to detect the end of the file before an error arises.

PRINT #

All data sent to a file with the PRINT # command is sent as ASCII characters. When you send numeric data such as PRINT #7;A,5,10 the numbers are spelled out as ASCII characters. These numbers can be read back using INPUT #7;A;B;C; or the data can be read back as string data: INPUT #7;A$;B$,C$; The error Nonsense in BASIC will occur if you try to read non-numeric data with a numeric variable input command.

File Records

A record is a text string terminated by a record separator, such as a carriage return (CHR$(13)) or a comma (CHR$(6)). The INPUT # command will read characters until it reads a record separator; then it will return with the entire record. It works the same as if you were using the standard INPUT command to read from the keyboard. It only completes input when you press enter. Also, there are certain characters that INPUT doesn't like, such as quotes «" "».

Sequential Files

Ordinary sequential files can use different length records within the same file. This type of file works well for data that does not need to be changed or updated, such as a spell checker dictionary file, or a database of history. But variable-length records are not very useful for a file or a database that needs to modify or update its records, because the start of a record can't be calculated if its position varies.

Random Access Files

For true random-access operation, the length of records should be the same for all records. This way, records can be easily replaced and modified, and files can be indexed for fast operations. To keep all records the same length, you just need to use a DIM array for sending data to the file. Data can be entered into the dimensioned array, and the same array is always PRINTed to the file.

To tell LKS/R the length of the file records (only needed when using the TAB and search command), we use the DATA command.

Example:

RANDOMIZE USR 26800; DATA channel, recordlength

Record lengths can be up to 2K but lengths of 32, 64, 138 or 256 bytes are more common.

PRINT # and INPUT # Command Structure

Care must be taken when using these commands not to send unwanted characters to the file. Always add a semicolon to the end of PRINT # commands unless you want a record separator sent.

INPUT commands can send data to a channel as well as read from it. For example, in the command INPUT "Enter Name"; a$ the INPUT command first sends the text "Enter Name" to the channel, then reads in a$, then sends a final CHR$(13) because there is not a semicolon after a$.

The correct syntax for INPUTing from a files is INPUT #channel; a$;.

The Use of LINE

The INPUT # command has the same problem as the standard INPUT command if quotes are entered as data. Because of this, the INPUT #channel; LINE a$ works better with data that has quotes in it.

Multiple inputs can be done in one command with INPUT LINE a$; LINE b$; LINE c$;

The LINE type input cannot be used for inputting numeric variables.

Records can be separated using carriage returns, apostrophes (') or commas.

Example:

PRINT #channel; "Record 1", "Record 2" ' "Record 3"

String Searching

Another variation of the DATA command is used as a search command to find a specific text string.

Example:

LET x = USR 26800: DATA channel, record length, "word to find".

This command will search from the current file position to the end of the file, looking for an occurrence of the specified text. If a match is found, x will be the number of records that the command had to search through to find the match. (It's the relative number of records from the initial position, not the absolute record number. To get the absolute record number, add the initial position to the return value x.) The search command can be repeated to find the next occurrences of the text string. If a match is not found, x will be 65535. The search command can only be used in random mode (RND). This command runs much faster than the equivalent BASIC routine.

PRINT TAB Command

After the record length has been set, the command PRINT #channel; TAB record number; can be used to randomly access any record in the file. The file pointer is set at the start of the record so the next record read or written will be that record. PRINT #channel; TAB 0; is an easy way to reset the pointer to the start of a file. Be sure to include the semicolon at the end of the TAB command, or a CHR$(13) will be sent to the file.

The TAB function allows for true random-access filing. This is something usually only found on very expensive computers. Using the TAB function and fixed length records, records can be updated and modified; and by specifying record lengths of one half or one quarter of the actual record length, parts of a record can be pointed to directly. If you try to access a record number that doesn't exist in the file a Number Too Big error will occur.

The LIST Command

The command RANDOMIZE USR 26800: LIST has been added for diagnostic purposes, to let you see what files and channels are currently open. The number in brackets on the far left of the screen is the buffer number. Buffer 1 is closest to the top of RAM, and they descend downward 5K per buffer as each is opened. The LKS/R will always try to use free buffers closest to the top of memory when a file is opened.

Buffers & Memory Usage

The LKS/R code loads in as a large REM statement, then moves the system variable PROG (23635) to point to the end of the REM statement. This makes the code invisible to BASIC and safe from everything except a NEW command. The length of the code is approximately 2500 bytes. The entry point for commands is 26800.

Example:

RANDOMIZE USR 26800: OPEN# 5, "file", OUT

The LKS/R uses a 5K (5100 bytes) buffer for each file open. These start at the top of RAM above RAMTOP and descend downward 5K for each additional file opened. When a file is opened, the LKS/R will use the first available buffer closest to the top of memory.

The system variable P_RAMT (23732), minus 256 bytes for UDGs, is used to find where the top of RAM is; so if you need some memory reserved at RAMTOP for machine code routines, you can poke P_RAMT down lower to preserve high RAM. Do this before opening any files.

You'll need to move RAMTOP with a CLEAR x command, with x depending on how many files you have open at once. When the LKS/R is loaded, it clears RAMTOP down low enough for two files to be opened. To calculate what value you should clear for the maximum number of files that you'll have open at once, use: P_RAMT - 256 - (1500 * number of files open).

It may seem that 5K per buffer is a lot of memory to give up. However, when you consider that this buffer will allow the T/S 2068 to access up to 800K of stored data per disk, then it's not such a bad compromise.

Configure Your Program to Auto-RUN

To use the LKS/R code with your programs, you need to have "SRcode.B1" load in. After it has initialized, it should then load in your program. This is easily done by adding a line after the initialization lines to load in your program. Then save the REM code to auto run at line 10. (See the LBASE.BC program for an example.)

Notes on Using Random Access / Sequential Files

  1. Be sure to close all files before removing a disk from the drive. If a disk is switched in a drive that has an open file the files on the disk will be corrupted!
  2. When using multiple drives, select the drive with the GO TO command in LKDOS before using the OPEN command. The LKS/R will remember the drive number after that for all files it accesses.
  3. Be careful not to accidentally use the LKDOS command RANDOMIZE USR 100: CLOSE #channel instead of RANDOMIZE USR 26800: CLOSE #channel to close random access files. The file may appear to be closed, but it really isn't. It does no harm to reissue the RANDOMIZE USR 26800: CLOSE #channel command if you suspect that a file is still open. It's a case of better safe than sorry.
  4. PAPER, INK and other attribute commands need to be used with care lest you corrupt your files. These commands send control bytes to the last stream opened.

    Example:

    10 INPUT #7; a$;
    20 PAPER 0

    Were you to run this, you would corrupt your file because PAPER 0 will send a control code to channel #7. To avoid any file corruption, use a PRINT; command before the attribute command to make the screen, which is channel #2, the last channel open, thus: 20 PRINT;: PAPER 0

  5. The ON ERROR command can be used to detect the End Of File and Number Too Big (TAB command) errors; however, the new error messages included in the LKS/R will cause the program to stop.
  6. Disks should have their write-protect stickers removed while files are in use. It's possible to read random access files while the write protect is on, but you won't be able to write new data to them.
  7. The LKDOS Version 3 EPROM has sequential file commands built into it. There may be some problems using the EPROM sequential file commands in conjunction with the LKS/R. This is because the EPROM-based filer keeps the file in the LKDOS cartridge RAM buffer as it is reading and writing to it. The LKS/R may at any time need a disk access. This will wipe out any data that is in the LKDOS cartridge RAM buffer. The LKS/R does not recognise the File Open condition that is in the Version 3 EPROM. This is because LKS/R is designed for use with any version including older versions of LKDOS that do not have the File Open condition.
  8. A single file can be up to 800K long; that is, it can take up the total capacity of a quad density disk. However, if a file becomes larger than 50 blocks, or approximately 250K, the directory entry that keeps track of the list of blocks will become so long that it will start to overwrite the next file name in the directory. This problem can be overcome by using any of the following strategies: 1) Don't have any files on the disk other than your random access file; 2) After creating your random access file, immediately add a few dummy files to reserve space for expansion; and finally, 3) You can use the LARKEN Disk Editor utility to enlarge the data file directory entry to the maximum size you'll need.

LBASE

An Example Database

What is LBASE?

On the disk is a simple database manager program that I've added to help demonstrate an application of random access files. I've also included a database of articles, news and reviews from the Timex/Sinclair user periodical Time Designs, issues January 1986 to September 1988.

Loading LBASE

To load LBASE, type RANDOMIZE USR 100: LOAD "LBASE.BC". This will load the LKS/R code, which will initialise itself and then load LBASE.B1.

LBASE Overview & Commands

The main menu should now appear on the screen. Type «O» to open a file. Enter the drive number then the file name «TIMEDESIGN». The file will be opened (as RND) at the first record. The remaining LBASE commands are:

«R» to read the next record,
«E» to edit a record,
«A» to append a new record,
«S» to search for a text string starting from the current record,
«C» to close the file, and
«N» to start a new file (though not on the master disk!).

Examining the BASIC Programming for LBASE

Break into the BASIC program and LLIST it to a large printer or simply LIST it on the screen to examine it. The program is not very long. Most of the programming is for the mini-editor and the menu print routines.

The key points of interest are:

  1. d$(256), which is the array used for sending and reading data from channel #7, the channel used for this file.
  2. The record length set by the DATA command is 257, because the record separator makes up the extra character.
  3. The variable CH7 is set to 1 if channel #7 is open, and set to 0 when channel #7 is closed. If other channels are open, you should use a similar value to keep track of their status as well.
  4. The variable SRDOS is used to make the listing clearer.
  5. The ON ERROR command is used to detect the end of the file. If it occurs, the file pointer is set to the beginning with the TAB 0; command.

Adding New Features to LBASE

There are many improvements that could be made to LBASE. A commercial database program like dBase for the IBM PC uses an index file as well as the main data file. The large data file is not in any particular order. The index file contains the record order in which the data file should be read. If you want, for example, to put the database in alphabetical order, dBase would read the entire data file though once and create a new index file accordingly. This way, sorts can be done much faster.

LBASE could have two or more files open at once, so records could be sorted to other files. This is another way of sorting the database.

Records can be inserted into the middle of a file by using two files. Open a new file as output; send a certain number of records to the new file; then send the inserted record or a blank record, and finally send the rest of the old records to the new file.

LBASE uses an unfielded record. Data is just entered in free form. Data fields could be added if required.

A more elaborate search command could be added – like the one in Profile®. One could add Boolean logic operators to do inclusive and exclusive searches. The programming for these new features could be done in machine code or compiled BASIC to improve speed.

Copying Files

Copying files is very easy in BASIC, although it may take some time. Here's an example:

10 RAND USR 26800: OPEN #6, "Source",RND
20 RAND USR 26800: OPEN #7, "Target",OUT
30 INPUT #6; LINE a$; : PRINT #7; a$

[TEXT APPEARS TO BE MISSING HERE]

The routine will return the character in the accumulator. A CHR$(226) signals that you've read the last byte of the file; or you could let a BASIC ON ERROR handle the end of file.

Once the address of the channel's input and output routines are found using #1230, these could be stored in memory by our routine so #1230 does not need to be called for each character. This would speed up the code significantly.

For keeping track of records, you need to keep a counter; or you could count file separators if you're sure that they're consistent.

Compiled BASIC

BASIC can be compiled to significantly increase the speed of sorting, copying and manual searching. However, the OPEN, CLOSE and DATA commands will have to be left in interpreted BASIC.

Technical Information on the Internal Operation

When LKS/R is loaded and initialised, the channel table is expanded by adding four new entries, for the maximum of four files that can be open. The entries consist of the output address, the input address and a one-byte device identifier.

The index register is used extensively in the code. The addresses of the file channels in the channel table point to entry points in the code, that just load the IX register with the address of the corresponding file information block, and then jump to the common code used for input and output for all files.

When a file is opened for output, if the file already exists, then the file information block (FIB) is set to the end of the existing data, and any new data is appended to the file. As each flock is filled, it's saved, the block number is added to the file's directory block list in the directory cell, and the block number of a new block is fetched from the track map, and marked as used in the track map. It's marked as used even before it's completely filled, because while the file is open, we don't want any other file to claim it.

When a file is opened as output, an error message will occur if a read attempt is made.

When a file is opened as RND, the FIB is set to point to the beginning of the file. The file can be read from or written to, so whenever the pointers reach the end of a block, the block is first saved, then the next block is loaded. Saving the block first ensures that any new data in the block is not lost. It's the reason for most of the disk activity as you read a RND type file. Some disk systems have a separate input-only file operation which runs faster. The sequential input in the V3 EPROM is for input only, so it can read a file much faster than the RND type in LKS/R.

Closing a file saves the block in the buffer to disk, disconnects the stream that points to the channel, marks the file closed in the FIB and, if the file type is output, enters the current block in the directory cell.

The search command tries to speed up disk searches. The current block is saved to ensure that no new or altered data is lost. Thereafter, it doesn't save any of the blocks while searching through them. This can't be done easily from outside the code, so external customised searches will be a little slower.

When the code sees a TAB character, it reads in the next two bytes to get the record number. It multiplies the record number by the length and then divides the product by 5090 to determine which block the record is in. The remainder is the offset into that block. The code then loads in that block and sets up the FIB to point to the block and offset.

Using a RAMDISK will increase the speed of file searches and file access significantly. The file can be copied to the RAMDISK for use and the copied back to disk when you're done.

Machine code copy routines could really speed up copying.


Page by William McBrine. Last updated: Apr 9 2000