Distributed Data Management Architecture (DDM) is IBM's open, published software architecture for creating, managing and accessing data on a remote computer. DDM was initially designed to support record-oriented files; it was extended to support hierarchical directories, stream-oriented files, queues, and system command processing; it was further extended to be the base of IBM's Distributed Relational Database Architecture (DRDA); and finally, it was extended to support data description and conversion. Defined in the period from 1980 to 1993, DDM specifies necessary components, messages, and protocols, all based on the principles of object-orientation. DDM is not, in itself, a piece of software; the implementation of DDM takes the form of client and server products. As an open architecture, products can implement subsets of DDM architecture and products can extend DDM to meet additional requirements. Taken together, DDM products implement a distributed file system.
Distributed applications
The designers of distributed applications must determine the best placement of the application's programs and data in terms of the quantity and frequency of data to be transmitted, along with data management, security, and timeliness considerations. There are three client–server models for the design of distributed applications:
File Transfer Protocol (FTP) copies or moves whole files or database tables to each client so they can be operated on locally. This model is appropriate for highly interactive applications, such as document and spreadsheet editors, where each client has a copy of the corresponding editor and the sharing of such documents is generally not a concern. Thin client applications present the interface of an application to users while the computational parts of the application are centralized with the affected files or databases. Communication then consists of remote procedure calls between the thin clients and a server in which uniquely designed messages specify a procedure to be called, its associated parameters, and any returned values. Fat client applications perform all application processing tasks on client systems, but data is centralized in a server so that it can be managed, so that it can be accessed by any authorized client application, so that all client applications work with up-to-date data, and so that only the records, stream sections, or database tables affected by an application are transmitted. Client application programs must be distributed to all clients that work with the centralized data.
The DDM architecture was initially designed to support the fat client model of distributed applications; it also supports whole-file transfers.
Benefits provided by DDM architecture
The DDM architecture provides distributed applications with the following benefits: Local/remote transparency. Application programs can be easily redirected from local data to remote data. Specialized programs that access and manage data in remote systems are not needed. Reduced data redundancy. Data need be stored in only one location in a network. Better security. By eliminating redundant copies of data, access to the data in a network can be better limited to authorized users. Data integrity. Updates by concurrent local and remote users are not lost because of conflicts. More timely information. Users of multiple computers in a network always have access to the most recent data. Better resource management. The data storage and processing resources of a network of computers can be optimized.
History
DDM architecture is a set of specifications for messages and protocols that enable data distributed throughout a network of computers to be managed and accessed.
Initial efforts
IBM's Systems Network Architecture (SNA) was initially designed to enable the hierarchical connection of workstations to IBM mainframe computers. The communication networks available at the time were rigidly designed in terms of fixed connections between a mainframe and its suite of workstations, which were under the complete software control of the mainframe computer. Other communications between mainframes was also in terms of fixed connections used by software defined for specific purposes. As communication networks became more flexible and dynamic, generic peer-to-peer communications were desirable, in which a program on one computer could initiate and interact with a program on a different computer.
When IBM's SNA Advanced Program to Program Communications (APPC) architecture was defined in the early 1980s, it was also apparent that APPC could be used to provide operating system services on remote computers. An SNA workgroup pursued this idea and outlined several possible distributed services, such as file services, printer services, and system console services, but was unable to initiate product development. APPC software was not yet available on mainframes and, more basically, mainframes were still viewed primarily as stand-alone systems. As a result, work on distributed services was suspended by the SNA work group.
Members of the SNA work group from IBM's Rochester, Minnesota development laboratory were convinced that a business case existed for distributed services among the mid-range computer systems produced in Rochester. A primitive form of distributed file services, called Distributed Data File Facility (DDFF) had been implemented to connect the IBM System/3, IBM System/34, and IBM System/36 minicomputers. Further, the IBM System/36 and the IBM System/38 computers were being sold to customers in multiples and there was a clear need to enable, for example, the headquarters computers of a company to interact with the computers in its various warehouses. APPC was implemented on these systems and used by various customer applications. The idea of distributed operating system services was then revived as the Golden Gate project and an attempt made to justify its development. This attempt also failed; the whole idea of distributed services was too new for IBM product planners to be able to quantify the value of software that interconnected heterogeneous computers.
However, one Golden Gate planner, John Bondy, remained convinced and persuaded management to create a department outside of the normal control of the Rochester laboratory so that there would be no immediate need for a predefined business case. Further, he narrowed its mission to include only support for Distributed Data Management (DDM), in particular, support for record-oriented files. He then convinced an experienced software architect, Richard A. Demers, to join him in the tasks of defining DDM architecture and selling the idea of DDM to the IBM system houses.
The first year of this effort was largely fruitless as the IBM system houses continued to demand up-front business cases and as they insisted on message formats isomorphic to the control block interfaces of their local file systems. Further, as Personal Computers began to be used as terminals attached to mainframe computers, it was argued that simply enhancing the 3270 data stream would enable PCs to access mainframe data.
During this period, Demers designed an architectural model of DDM clients and servers, of their components, and of interactions between communicating computers. Further, he defined a generic format for DDM messages based on the principles of object-orientation as pioneered by the Smalltalk programming language and by the IBM System/38. This model made it clear how DDM products could be implemented on various systems. See How DDM works.
In 1982, the System/36 planners became convinced there was a sufficient market for DDM record-oriented file services.
DDM level 1: Record-oriented files
The generic format of DDM messages had already been designed, but what specific messages should be defined? The System/36 file system had been defined to meet the record-oriented needs of third generation programming languages (3GLs), such as Fortran, COBOL, PL/I, and IBM RPG, and so had the System/38 file system and the Virtual Storage Access Method (VSAM) file system of the IBM mainframe computers. And yet, their actual facilities and interfaces varied considerably, so what facilities and interfaces should DDM architecture support? See record-oriented files.
The initial work on DDM by the Golden Gate project had followed the lead of the File Transfer Access and Management (FTAM) international standard for distributed files, but it was very abstract and difficult to map to local file services. In fact, this had been one of the barriers to acceptance by the IBM system houses. Kenneth Lawrence, the system architect responsible for System/36 file services, argued that it would be better to define messages that at least one IBM system could easily implement and then let other systems request whatever changes they needed. Naturally, he argued for support of System/36 requirements. After a year of failure to sell the idea of DDM to other IBM system houses, Lawrence's arguments prevailed.
Richard Sanders joined the DDM architecture team and worked with Lawrence and Demers to define the specific messages needed for System/36 DDM. Progress in the definition of DDM encouraged System/38 to also participate. This broadened the scope of DDM record-file support to meet many of the requirements of the System/38's advanced file system.
Files exist in a context provided by an operating system that provides services for organizing files, for sharing them with concurrent users and for securing them from unwarranted access. In level 1 of DDM, access to remote file directories was not supported beyond the transmission of the fully qualified name of the file to be used. Security and sharing, however, were required. Sanders did the design work in these areas. Sanders also defined specific protocols regarding the use of communication facilities, which were incorporated in a component called the DDM Conversational Communications Manager. Initially implemented using APPC, it was later implemented using TCP/IP.
With the completion of the System/36 DDM product, Lawrence worked with programmers from the IBM Hursley Park, UK laboratory to adapt much of the System/36 DDM server programming for use in the IBM Customer Information Control System (CICS) transaction processing environment, thereby making CICS a DDM server for both the MVS and VSE mainframe operating systems. Lawrence also worked with programmers from the IBM Cary, North Carolina laboratory to implement a DDM record-oriented client for IBM PC DOS.
Level 1 of DDM Architecture was formally published in 1986. At the time of this announcement, IBM presented an Outstanding Technical Achievement Award to Kenneth Lawrence, an Outstanding Contribution Award to Richard Sanders, and an Outstanding Innovation Award to Richard Demers.
In this article, System/38 will be henceforth used to refer to the System/38 and its successors: the IBM AS/400 (which merged the functionality of the System/36 and the System/38), the IBM iSeries, and the IBM Power Series (which merged the iSeries with the IBM RS/6000, IBM's RISC/UNIX-based server and workstation product line).
DDM level 2: Hierarchical directories and stream-oriented files
With the increasing importance of the IBM PC and the Unix operating system in network environments, DDM support was also needed for the hierarchical directories and stream-oriented files of the IBM Personal Computer running IBM PC DOS and the IBM RS/6000 running IBM AIX (IBM's version of Unix). See Stream-oriented files.
DDM Architecture Level 2 was published in 1988. Jan Fisher and Sunil Gaitonde did most of the architecture work on DDM support for directories and stream files.
DDM level 3: Relational database services