Interend Protocols
This document discusses some of the generally available end to end protocols of IPSANET.
TEMPORARY PREFACE for reviewers:
1] this document doesn’t say anything about X25 or SDLC. The reason is that I don’t anything about those areas. The only thing which I could say that the Alpha viewed the X25 connection as a group of virtual calls whereas the SDLC connection was viewed as a strict point to point protocol.
2] Around 1983 or 84 SCR made changes to the alpha support for ASCII terminals. He introduced a protocol such that an APL function using ARBOUT could alter the behavior of the Alpha with respect to input. This is interesting because it bypasses the 3705. If I had user documentation for those changes, I could describe them here.
3] I have deliberately omitted Telex access because it was implemented via a standalone computer rather as a new LO terminal type (see Roger Shaw’s paper on the website for discussion).
4] The Net90 project identified certain missing features in the interend protocol. The most universal of these is described in David’s INSITE/SEPT 86 article: “Automatically re-establishing a user’s network signon call to a new route, if available, should the old route fail.” Perhaps I should include some discussion this (particularly why it wasn’t feasible with 64k storage in the endnodes.)
/RDM 30 July
Common interend
APL T-task support
Telex pipeline
Dialin Bisync including direct printer/tape connection
Dialout Bisync and async
NSVP to NSVP call
X.25 interface
SDLC pipeline
Interend packet format
STX <count> <priority etc> <address one> <address two> <interend byte>
<zero to 53 bytes of data> ETX CRC1 CRC2
<priority etc> contained three fields: abbreviated node number, internode sequence, and forwarding priority. The first two fields were strictly for data link control. The forwarding priority field originated in the interend software and could assume values 1 2 or 3. (Zero indicated a data link control packet.)
<address one> & <address two> are discussed in the route control essay.
<interend byte> contained three fields: one unused bit, five bit interend sequence, two bit interend type. The interend sequence number was assigned by the packet sender. The other end processed received packets in sequence rather than in order of receipt. This protected against the possibility that data link control might forward packets out of order due to transmission errors. <interend type> had three legal values: control, normal, turnaround.
Interend control packets used two data bytes (one 16 bit word). The first byte was control type, second byte was unused in except in BPR packets. The four types were: break, break acknowledge, soft drop, back pressure relief (aka BPR).
Some interend capabilities were used only a few subprotocols. APL T-tasks used turnaround and break. Network bisync used turnaround but not break. Soft drop and BPR had more general use.
APL T-tasks
{RDM: missing stuff which may not belong here:
1] Alpha autospeed and host selection processing
2] Byte reversal in 3705 and APL}
The primary use of the network was to allow an asynchronous terminal to connect to a T-task on a Sharp APL system. (T-task was the only kind of task supported by the original APL\360 software.) Communication between a terminal and a T-task is strictly half-duplex. At any time only one end is allowed to transmit; the other must remain silent with the possible exception of a break signal. This worldview originated with IBM hardware (the 1050 and 2741 terminals). It carried over into the IPSA/STSC support of ASCII devices.
When the Alpha received characters from a terminal it examined them for a turnaround character. CircleC signalled 2741 turnaround; carriage return was used for ASCII devices. The current buffer of received characters was completed as a turnaround packet and forwarded to the inboard end. The Alpha then ignored characters from the terminal until it received a turnaround packet from the inboard end. After all of the characters in the packet were sent to the terminal, the Alpha was again ready to receive characters from the terminal. Received characters were stored in a buffer. When the buffer was almost full (about ten bytes remaining), a medium priority task was started to send a non-turnaround packet and get a new buffer. It was theoretically possible for a few more bytes to be received and stored before a new buffer was obtained.
The turnaround packet from the 3705 (inboard end) normally contained a final character to the terminal to indicate turnaround. For a 2741, CircleC physically unlocked the keyboard. ASCII bell was used an invisible turnaround indicator. There was some controversy when the bell was introduced but it remained the convention for many years.
The APL user was sometimes dissatisfied with the output arriving at the terminal. The usual causes were no output or too much output. In both cases the user signalled break by pushing a key (2741 Attention or ASCII break). The outboard end decoded the somewhat messy break indication and sent an interend break packet to the inboard end. The outboard end then started discarding data packets from the inboard end until a break acknowledge packet was received. The break acknowledge indicated that the inboard end had received the break indication that subsequent output was generated by APL in response to the break.
Soft drop in the original protocol was rather elegant. It was used to request a disconnect after the preceding data packets had been delivered. The most common use signalling to the outboard end as when processing an APL )OFF command. This command indicated that the user wished to disconnect. Some usage statistics were sent to the terminal before the soft drop indication. When the outboard end processed the soft drop, it sent its soft drop in response.
Another use of soft drop was when the Alpha detected disconnection of a switched telephone connection. It monitored dataset signals from the answering V.21 or 103A2 modem. Loss of data set ready (DSR), clear to send (CTS) or carrier detect (CD) for several milliseconds was deemed a loss of connection. This generated a soft drop packet to the inboard end. When the inboard end sent a soft drop in response, the Alpha could clean up the situation at the modem and allow a new call to be established.
The elegance of this symmetric auto-acknowledge protocol was if a race occurred where both ends sent soft drop simultaneously. Then each end treated the received SD as an acknowledgement to the SD which it had sent. When the all virtual call routing scheme was introduced, the symmetric drop protocol had to be replaced. The problem was that the intermediate nodes had to know that a virtual call was being terminated. The old soft drop was replaced by a request for a hard drop from the remote end. This allowed buffered output to be sent to the terminal before the telephone call was disconnected.
The original protocol had a backup to soft drop. If the soft drop was not acknowledged in one minute, a hard drop was sent. A hard drop was a node to node message rather an end to end message. It bypassed interend sequence number processing.
Back pressure relief was an end to end flow control mechanism with rather eccentric rules. It was used for T-tasks and the various network bisync protocols. The use in T-tasks was asymmetric: the inboard end was constrained by flow control but the outboard end was not. This resulted in a small gain in efficiency at the price of some protocol complexity. The arguments regarding BPR protocol choices and bandwidth efficiency are discussed with some rigour below.
An end that was subject to flow control had an integer variable called back pressure relief counter (BPRCNT). When it began transmission due to receipt of turnaround or protocol initialisation rules, it jammed BPRCNT to an agreed positive value. Before the transmitter tried to fill another data packet, it first checked to see if BPRCNT>0. A nonzero count allowed the transmitter to decrement BPRCNT and send one data packet. IF BPRCNT was zero, the transmitter waited for back pressure relief from the other end. When a BPR packet arrived, the second byte was added to BPRCNT and transmission could resume.
For 30cps or slower terminals, the initial BPRCNT was two. When the Alpha received the first data packet it sent a BPR with count of one. Assuming a single satellite hop and data available at the APL host, the third data packet should arrive about 700 ms after printing of the first packet began. When the printing of the first packet is finished there should be two more packets of 53 characters each queued at the Alpha. This allowed smooth printing at low speed.
When 120cps terminals were introduced, it was obvious that having 106 buffered characters was not enough to prevent pausing between packets. A BPRCNT of eight was used. The 3705 sent 120cps data packets at priority two rather than the priority one used for 30cps terminals. The intent was to keep 120cps terminals from hogging all the bandwidth. Also when 120cps terminals were introduced the 3705 software lacked tertiary flow control. Tertiary flow control restricted the number of packets queued for transmission from the 3705 to one per virtual call. With the restrictive data link connected directly to the 3705 this divided the bandwidth evenly. As the population of 120cps increased there was pressure to change the forwarding priority to one so that 120cps terminals would never run slower than 30cps terminal. This was done.
Network bisync used an initial BPRCNT of twelve and forwarding priority three. This was fairly satisfactory with a suitable software configuration on the customer end. Under heavy network load it was possible for a bisync call to be completely locked out.
BPR choices versus Bandwidth Efficiency
The choices associated with BPR rules affect bandwidth efficiency. In calculating efficiency, I assume:
OL is the mean output length in response to a short input.
NP is the number of packets required to send OL characters
PL is maximum packet length (total number of bytes sent)
Internode acknowledgement packets always hold two ACKs (3705 heavy load rule)
Input always fits in one packet (ie is less than 21 characters in 32 byte packet scheme).
Outboard node either sends BPR with count of one (Alpha rule) or (3705 rule)
Short input pkt and turnaround pkt never or always elicits BPR from other node.
remarks on the APL expressions which calculates bandwidth efficiency:
6.5 is cost to acknowledge one received packet
ceiling NP div 1 2 choice is OB end sending BPRCNT of 1 2
-1 0 adjustment to count of BPR pkts is never/always
0 13 choice is input/turnaround BPR never/ always choice
NP := ceiling OL div PL-11
BE := 6.5(OL+11xNP)+:2 2 rho 0 0 13 13)+
6.5x (–1 0) null .+ ceiling NP div 1 2
[Never is the BPR protocol as implemented; Always assumes two minor protocol changes.]
[&1 assumes BPR 1 for every pkt; &2 assumes BPR 2 for alternate pkt]
Four numbers are for output length of: 50 100 1000 10,000
Results
|
PL |
Never&1 |
Never&2 |
Always&1 |
Always&2 |
|
OL |
50 1E2 1E3 1E4 |
50 1E2 1E3 1E4 |
50 1E2 1E3 1E4 |
50 1E2 1E3 1E4 |
|
32 |
.488 .533 .543 .545 |
.521 .573 .594 .595 |
.410 .483 .538 .544 |
.433 .515 .587 .595 |
|
64 |
.741 .741 .750 .751 |
.741 .778 .785 .788 |
.575 .647 .740 .750 |
.575 .676 .773 .786 |
|
96 |
.741 .741 .826 .829 |
.741 .778 .854 .856 |
.575 .647 .813 .828 |
.575 .676 .840 .855 |
|
128 |
.741 .851 .864 .869 |
.741 .851 .884 .891 |
575 .730 .850 .868 |
575 .730 .869 .889 |
Comments on the above table:
IPSANET as implemented in 1980-1989 represented by column “Never&1” and row 64. (Maximum packet length was changed in 197?). Thus bandwidth efficiency in the mature IPSANET was from 74% to 75%. It could have been about 3% better with a slight change in the Alpha emission of BPR. Eliminating BPR jamming on turnaround would have had a noticeable effect on 100 character output line efficiency (compare PL=64 Never columns with Always&2).
Telex pipeline
Massey Ferguson wanted to save money on their dedicated Telex circuits. They had a message centre in Stoneleigh England. There were two Telex circuits from Toronto and several from the continent. A variant of the Alpha T-task protocol allowed a pair of Alpha ports to act like a dedicated Telex circuit. There was no flow control and no break signalling. Call establishment worked by designating one port as the outboard line and the other as the inboard line. Every minute, the outboard end checked to see if any configured outboard Telex lines needed to have a call established. If so, an IBA was sent to the inboard port associated with the outboard line. The scheme was fairly simple and robust. In theory communication was full-duplex but this probably not true in practice. The reason that a Telex circuit usually has only one current loop used for both transmission and reception. Either end can send characters by rapidly opening and closing the loop. If both terminals attempt this simultaneously, garble will result.
There was a minor problem when the UMM (Universal Modem Multiplexor) was first used with Telex lines. The AMD 9551 chip behaviour when receive data was in space hold caused a problem. Null characters were received faster than the remote end could transmit them. This caused buffer overflow in the other node. The solution was to search for a data buffer in which all characters were zero. Such buffers were discarded rather than transmitted.
Dialin Bisync including direct printer/tape connection
The original Bisync support allowed a remote terminal with a synchronous modem to dial a similar modem connected to an Alpha and exchange files with the SHARP APL host. The type of file which could be sent or received was severely constrained by the properties of the terminal. Devices that emulated the IBM 2780 or 3780 were an important category. They could receive print lines or exchange eighty column card images. Some other devices communicated via Bisync blocks with a maximum size of 512 bytes. Remote printing and several file transfer applications were supported within this framework.
Data flow was strictly half-duplex with a long sequence of blocks sent in one direction before line turnaround. Data packets were always subject to flow control; break was unsupported.
Call establishment occurred after the Alpha received the first data block (which could be empty). A bisync port had a fixed inboard node number to which it sent an IBA/IBR. When the 3705 received a request for a new call of type bisync, it used a special protocol to communicate with SHARP APL. It had several properties:
A] No byte reversal on data exchange between 3705 and APL
B] Automatic signon (without password) of an R-task (T-task variant) on user number 1001.
C] 3705-supplied APL command: )LOAD 970 BSI
D] Autostart of the loaded workspace quickly lead to an ARBIN so that the R-task could receive the initial transmission from the Alpha.
E] The R-task expected an initial sequence of one or more EBCDIC card images providing information about device type, billing, file names, data reformatting scheme, etc. If the initial transmission from the remote Alpha was empty, the R-task assumed that it was communicating with a hard-wired device of some kind. It used the remote node number to index a table of command file names. R-task commands were read from an APL file.
In 1978?? Steve Crouch wrote a version in which the support for a true bisync device was replaced by code to connect a medium speed printer via asynchronous port two. A further extension allowed an industry-standard nine-track tape drive to be attached to the Alpha.
Data packets sent between the bisync Alpha and the R-task used a special format known as “network bisync”. This protocol provided simple data compression and some inband signalling. There were three types of control determined by the top bits of a byte:
0-63 range: N+1 data bytes followed
64-127 range: single byte follows. It is to be repeated (N+1) times.
128-191 range: type two control. Always a single byte.
192-255 range: illegal
Control phrases were not allowed to span a packet. The R-task was ignorant of network packet size. The 3705 had the duty of resolving this by splitting some type zero phrases into two phrases to avoid straddling a packet. Similarly a type one control was not allowed as the final character in a packet.
The type two controls CITB, CETB, CETX, CEOT were isomorphic to bisync controls of similar names. There was also an indicator that the transmission was in transparent bisync mode rather than normal. The R-task inserted this before the first block which contained characters forbidden in normal bisync. The Alpha inserted this before the first transparent block which it received.
Dialout Bisync and async
One hardware development in the 80s was the development of alternatives to the Western Electric 801 Automatic Calling Unit (ACU). The 801 allowed a telephone call to be established with a computer supplied number. It had an interface which was quite different from RS-232. Only computers equipped with suitable hardware could communicate with the 801. As telephony regulation loosened in the United States and Canada, easily usable autodiallers became available. IPSA purchased one which accepted call setup instructions using asynchronous ASCII and then switched to a pass-through mode with the data terminal attached to some external modem (a synchronous modem in this case). As the UMM could operate in both asynchronous or synchronous modes, it was theoretically possible to dial out from an Alpha to a bisync host.
SHARP APL TSIO already contained a feature which allowed some users to directly control a channel attached device. This was initially introduced to allow an APL program to load or dump a 3705 via the NSC address. The APL configuration was modified so that some ESC addresses were left used by APL. The 3705 software was modified so that a bisync call could be established from the channel side. The APL program sent IBR parameters and a protocol indicator. (This scheme trivially provided communication with another SHARP APL system using T-task protocol.) Some new type two controls were defined in the network bisync protocol. These allowed the APL program to control the autodialler. After successful establishment of the telephone call, the APL program could send a sequence of bisync blocks to the remote machine and receive another sequence in response.
With the same autodialler hardware, an asynchronous modem could be connected and communication via the normal T-task protocol was possible (no break support).
NSVP to NSVP call
Network Shared Variable Processor was a scheme for sharing variables between two different APL systems. (IBM’s APLSV introduced shared variables. Paul Berry discusses shared variables in Chapter 31 of the “SHARP APL REFERENCE MANUAL”.)
In the original NSVP implementation, the primary IPSA APL host had one NSVP N-task for every remote destination. Several remote shares (established from either end) were multiplexed over an NSVP call. This network NSVP call was relatively permanent in that when the N-task was started, it periodically attempted call setup. Subsequent work by Richard Potyok replaced the APL N-tasks with 370 assembly language code.
The network NSVP call was remarkable in one respect. It provided full duplex communication between a pair of 3705s. This was accomplished by using two ESC addresses. One ESC address was transmit-only; the other was receive-only. Communication used the network bisync protocol with a greatly enlarged set of type two controls. These new type two controls were strictly NSVP protocol and the 3705 understood only end-of-block.
For transmit the 3705 broke up a large (1500byte) block of network bisync phrases into packets using the rule of not splitting a phrase over a packet boundary. I think expiry of the write CCW count forced a short data packet (this differs from T-task and Bisync support where write command boundaries were ignored). For receive the 3705 had to decode the type two control which specified end-of-block and signal an end to the read CCW.
There were other variants of NSVP protocol. There was a VTAM to VTAM version. With a slight change to call setup, the 3705 NSVP could place a call to an Alpha X25 Gateway. If a VTAM NSVP call routed out of SNA via NPSI was waiting in the Alpha, the two would be joined. This allowed an APL hosted served by an IPSA 3705 to communicate with a VTAM/NSPI APL host.
X.25 interface
SDLC pipeline