Stowe Boyd is director of Research and Development at Meridian Software Systems where he is currently developing the Meridian Software Development Platform, a distributed environment based on HyperText principles. He has published numerous articles on the subject of software development environments and related topics.
Some programmers argue that contemporary programming languages are simply inadequate for developing large software systems. They maintain that current implementations of programming languages may actually hinder more than help in the construction of reliable, verifiable, or even manageable software. Their point is that any software development component, no matter how good it is by itself, is compromised if it doesn't work well in combination with other tools. This perception has led to the development of integrating/integrated software, such as code documenters, analyzers, librarians, interactive debuggers, and more, designed to address the problems of large scale software development.
In his paper "Beyond Programming Languages,"{1} Terry Winograd states that "the main activity of programming is not the origination of new independent programs, but the integration, modification, and explanation of existing ones." Winograd goes on to point out that the intellectual activity associated with software development is largely one of gaining insight. Only a small proportion of time is dedicated to generating new software, especially when compared to all other activities. This is not a failure of environments or programming languages but rather an inescapable reality.
At some ill-defined point, a set of programming tools that offers a sufficiently high level of integration may become collectively known as a software engineering environment. This article examines a handful of development environments that were selected on the basis of their historical impact, their degree of use, or their promise for the future. These criteria are examined in light of three key themes: point of view, structure, and scope.
A software development environment exists to increase the programmer's awareness of the program under development. This understanding may be limited to the static and dynamic aspects of the software backplane or may extend beyond that, representing a model of the software development process itself, including development phases, organizational structure, and the interrelationships between projects.{2}
Ultimately, the user's point of view dominates all evaluation of environments. The bias expressed in this article is a corollary to Winograd's observation that "An environment serves to the degree that it assists in the reuse of software first by providing insight into the software and its workings and structure, and then by providing means to modify and manage software systematically and the environment's success is directly proportional to the naturalness, uniformity, and generality of the user's point of view."
Users' concerns focus on exterior aspects of a system, such as the user interface, performance, capabilities, and user modifiability. But there are other perspectives. Tool builders, for example, have broader concerns, which revolve around system structure. Management evaluates environments based upon both exterior aspects and system structure because these characteristics influence flexibility and productivity, but they are more concerned with the scope of the environment.
The natural complement of point of view is environment structure, which reflects the system's architecture. An environment's point of view sometimes provides a limited reflection of the system's structure. In general, however, this is not the case -- in fact much of what goes on "below the hood" is never seen by the user.
Environment components form a framework that supports software development. The effectiveness of a framework cannot be judged abstractly. First, it must contain the required tools. It must also be robust enough to support software while it is undergoing development. Finally, it must focus user activity in ways that contrive to enforce the project's organizational requirements. This dimension -- the environment's dynamic nature -- is environment scope.
Ask any three software engineers if software has a life cycle, and they would probably agree that it does. Ask them to describe the life cycle, and you will probably receive three different answers. Their answers would depend upon many factors: organizational policies, training, experience, and psychology. It is unlikely that you will receive one expressive, generalized definition of the software life cycle.
Still, software is both the result and a record of the complex development process. This process comprises, but is not limited to, activities such as design, coding, testing, and documentation. The span of activities that are supported by an environment thus defines its scope. For example, an environment that provides capabilities for automated test generation and measurement is likely to lead to the development of higher quality software than one that does not. Scope, however, trails point of view and structure in importance for the following two reasons:
Describing environments can be something like dissecting small animals: too many cuts and there is nothing left to examine. For this reason the following discussion addresses the major trends in environment research over the past few decades, making only a few cuts into the specific environments mentioned and focusing on major contributions.
Early research and development in environments went on for some time prior to the adoption of the term environment. These early efforts were generally concerned with a specific problem such as the development of system software in the Unix system,{3,4} experimental programming in Cedar/MESA,{5,6,7} or exploratory programming of Lisp programs in Inter-Lisp.{8}
It has often been noted that everyone has a programming environment; generally it is ad hoc and not coherent. If an operating system with a bunch of tools is an environment, why go any further? One example of an operating system that has gradually evolved into an environment is the Unix system and its many descendants and variants.
The Unix operating system{3} was invented in 1969 by Ken Thompson, and by 1979 it was in use in more than 2,300 computers.{4} Today, there are literally hundreds of thousands of Unix systems in use in the world, and it has become not only a de facto operating system standard but also a proposed standard.{9}
The Unix environment was originally devised to support tool building, and therefore its point of view is that of a tool builder, not a general software developer. Its basic structure is largely visible to the user and has been characterized as follows:{4}
More abstract reuse occurs in the form of code generators, such as YACC and Lex.{10,11} Software designers can apply these source code generation tools to the tasks of parsing textual input simply by specifying the input's form. This form of reuse support has driven work in diverse areas such as compilers and user interface systems{12,13} and has structured language-specific environments within Unix.{14} The Gandalf project at Carnegie-Mellon{15} and the synthesizer generator project at Cornell{16} have built upon these ideas. This has led to syntax-oriented environments in which editors, compilers, and other tools share a common representation of software.
These ideas continue to be applied today in projects such as the Distributed Ada Programming Support Environment effort{17,18} and the Anna project,{19} which are dedicated to distributed development and the formal specification of Ada systems, respectively.
The Unix filter paradigm of program construction -- small, single-purpose programs linked by I/O channels -- has had a major influence on tool building. The ODIN/Toolpack{2O} and Arcadia{21} projects have adopted the filter approach and extended it by considering environment tools as assemblages of tool fragments that are brought together for a specific purpose and linked by interprocess communication channels.
The original Source Code Control System (SCCS){22} and the related Make program{23} defined an extremely serviceable model for the specification of software system composition. The SCCS system is a Unix toolset that manages versioned text files, provides the user with a means to incorporate changes into software systems, and manages related documents. The model is hierarchical in that changes can be made away from an arbitrary base-line; a version-numbering scheme is provided based upon the fundamental tree form. make is a tool that derives executable software systems (or components) based upon a specification (the "make-file"). This specification has two classes of definition:
In response to the problems inherent in the SCCS/Make approach, an enormous amount of work has gone on in the area of software composition. Revision Control System (RCS){24} is a counter to the incompatibilities in SCCS/Make that incorporates several more efficient and powerful features, such as backward version maintenance, retrieval of file versions by multiple keys, and deferral of specific versions.
Apollo's Domain Software Engineering Environment (DSEE){25} represents a major departure from the basic SCCS/ Make approach. DSEE supports programming in the large across a distributed network of Apollo computers. Software composition in DSEE distinguishes the hierarchic system model from the configuration. Composition is source-limited; all objects are managed transparently by DSEE in an "object pool" that is shared among users. Versions and variants of software are maintained separately. Versions are more or less equivalent to SCCS or RCS versions, but DSEE variants also have novel features, including alternate implementation of system elements (different floating-point libraries for different machines, for example) and conditional compilation.
The actual construction of a DSEE system binds a system model to a specific version set using a "configuration thread" (CT). A CT comprises a series of rules for determining the actual set of versions to be used; evaluation of the rules results in a bound configuration thread. It is this bound form that derives the actual executable software. Shared objects appropriate for inclusion in the executable are found prior to duplication, minimizing construction effort.
DSEE shifts the Unix point of view from a tool builder's to a true software developer's. Other more recent efforts, such as the Sun Network Software Environment (NSE),{26} represent the awareness of the need to support software composition in a uniform, reliable, and robust way without forcing the developer to be a tool builder. This system automatically updates information used to manage system composition and, like DSEE, closely integrates version and variant control with system build activities.
The second and potentially greatest effect of the widespread adoption of these systems is a shift in the Unix structure. NSE extends the Unix information base to include versions and several other structural changes of significant scope. These represent a point of departure from the traditional Unix paradigms of TTY-type terminals, simple time sharing, and the filter approach to information sharing across tools. Unix has made a large contribution, but it is not enough.
Unix is an environment geared to the composition of small- to medium-sized systems and the construction of tools. Efforts such as DSEE and NSE represent a new phase for Unix as a software engineering environment for large systems. It is noteworthy that Unix software composition capabilities are sufficient to manage the complexities of Unix itself -- all 500,000 lines of it -- and have proven useful for the management of medium-sized business software systems as well.{27} When confronted with systems comprising millions of source-code lines, however, other approaches are required.
The Ada Language System,{28} the WIS Software Development and Maintenance Environment,{29} and the Rational Environment{3O} are attempts to support large-scale development for Ada systems. Ada is designed for the construction of large, reliable, maintainable, embedded systems and needs the support of an appropriately devised environment.{31} Note that this is not the fundamental character to the language but an aspect of the character of large systems.
Lisp is one of the earliest programming languages and one of the most hardy.{32} Sandewall{33} characterizes the use of the language quite well: "Lisp is used almost entirely as a research tool. The average Lisp user writes a program as a programming experiment, that is, in order to develop the understanding of some task, rather than in expectation of production use of the program. The act of developing the program, not the act of running it (even for test data) constitutes the experiment. As a consequence, the program is likely to be large and complex, to undergo drastic revisions while it is being developed, and to be thrown away before it has been 'completed' by conventional programming standards since it will already have served its purpose before then."
Very early it became evident that experimental programming constituted a radically different point of view than was then current. Indeed, in 1969 Teitelman{34} introduced the concept of a programming environment: "The programmer's environment influences, and to a large extent determines, what sort of problems he can (and will want to) tackle, how far he can go, and how fast he'll get there. If the environment is cooperative and helpful (the anthropomorphism is deliberate), the programmer will be more ambitious and productive. If not, he will spend most of his time and energy fighting a system that at times seems bent on frustrating his best efforts."
Lisp users were atypical at that juncture: they were committed to increasing programmer productivity by the application of computer resources and believed in the development of sophisticated tools rather than simple, multipurpose ones. This was both because of the point of view of the developer as experimenter and the intense scope of their programming environment.
The single most influential experimental environment effort of the 1970s was InterLisp.{34} Founded at Bolt Baranek and Newman, and later pursued at Xerox PARC, InterLisp represents the expert-oriented environment. The environment is geared to exploratory programming in Lisp, but the most important contributions of the work are not Lisp specific:
Another branch of research at Xerox has led to a worldwide revolution in computing. Smalltalk, a programming system that has evolved over a decade into Smalltalk-80,{38,39} is based upon the principles of object-oriented programming: objects and messages.
As defined by Goldberg,{39} "An object is a uniform representation of information that is an abstraction of the capabilities of the computer to store information. An object has the capacity to store information; we say that an object has a private memory. An object also has the ability to manipulate its stored information or to carry out some activity. These are called the operations of the object. The set of operations is referred to as the object's interface."
Objects communicate by message passing: "An object carries out one of its operations when another object sends it a message to do so. Each object knows the messages it can understand; associated with it is a procedure or method that describes how the object should answer the message."{39}
However important object-oriented programming may be or may come to be, the Smalltalk point of view has been much more influential. Smalltalk provides multiple views of the software under development. These include browser views (used to read object descriptions, to modify operations, and to perform change and version management) and inspector views (for discovering the state of objects during debugging).
Smalltalk's scope is tightly coupled to the exploratory development of Smalltalk-80 programs. Reuse of existing software is supported in a complex and powerful manner. All Smalltalk-80 objects are instances of an object class. A class is an abstraction that allows the description of common characteristics. Smalltalk-80 programming consists of creating classes and their instances and specifying sequences of messages to these instances.
New classes are generally derived from existing classes by inheritance of the parent classes' characteristics with new extensions. The purpose of much of Smalltalk-80 is supporting users in understanding existing class definitions (built-in explanations and example messages, for example) and extending them in novel ways.
Coupled with high-resolution workstation technology, Smalltalk's point of view has evolved into the standard spatial metaphor, as popularized by the Macintosh. Small but informative touches such as the changing of the cursor shape to denote current activity are used in Smalltalk-80 to provide consistent feedback to the user. The development of "nonintrusive" editors allows a nonmodal form of interaction. The user points to an area in which something should happen, selects that area, and then chooses an operation from a menu. Simply by pointing to another area, the user can signal a change about what actions to take.{39}
In retrospect, Smalltalk-80 is the source of the contemporary lingua franca for user interaction with environments and for computer interaction in general. The concept of multiple views over a collection of software components is perhaps the most basic information structuring concept to come from this work. Indeed, it has led to the development of conceptual programming systems.{40,41}
Although Smalltalk-80 permits the exploratory development of relatively complex systems, the complexities introduced are generally structural -- class hierarchies and complex message mechanisms -- rather than complexities because of size.
Cedar is an environment research project at Xerox designed to support exploratory programming.{5,6,7} The project was not principally oriented toward environment research, and therefore its structure is often conservative in design. Although it was not intended for very large systems, several of its principal structural features in fact mirror the requirements for programming in the large.
Cedar is both an environment and a language; the language is a superset of the Mesa language. Mesa introduced a few language features that are fundamental to the development of reliable large systems, most notably the separation of interface and specification of programming modules. Cedar differs from Mesa primarily by the inclusion of collectible storage, freeing the programmer from responsibility for management of free storage. These attributes are critical because Cedar is basically an open system and all applications operate in a single, large address space.
Cedar represents one of the first environments in which a significant commitment was made to capitalize on each developer having a high-performance, high-resolution workstation. This is a central component of the environment structure. Just as important to the user's perception of the environment is the ability to "multitask" and to switch rapidly among several different tasks. The Cedar system allows for the efficient creation and management of many independent processes, and system applications are designed to be nonintrusive so that users may determine the most effective use of their time.
Several other aspects of the system structure deserve mention:
Arcturus{44} is an environment project geared to programming in the large. Like Cedar, it is based upon an extremely modular language, Ada; is designed for high-performance, high-resolution workstations; and permits a wide range of user-definable presentation mechanisms to help users understand the purpose of source code. Arcturus accommodates the point of view of the large-system builder and at the same time allows for rapid prototyping of programs, based upon the ideas of Lisp Programmer's Assistants, as in InterLisp.
Note that Arcturus is founded upon the notion that "paper is the wrong container for documentation." Program documentation is considered as critical in Arcturus as in Cedar: "... documentation must play different roles for different audiences. Depending upon the experience and knowledge of the reader, documentation should reveal appropriate facts -- what is appropriate to one reader may be either boring, obvious, and condescending to another, or completely beyond the intellectual grasp of another.
"What is needed is a database in which program forms at various levels of refinement have attributes which lead to comments, whereupon various computed views can in turn make these comments selectively visible."{44}
Programming in the large is the clearest example of where contemporary environment technology fails. Despite the efforts of hundreds of researchers and the expenditure of hundreds of millions of dollars, no solution to the problems encountered with large systems has been discovered or invented.
There is promise in new and novel approaches and in the coupling of these emerging approaches with traditional mechanisms. Just as encouraging is a growing awareness in the boardrooms of America that software requires capitalization, that it is not developed in an assembly-line fashion, and that the primary cost factor of software is maintenance.
Application of widely accepted standards to software engineering environments will have a major influence on software productivity. Not only will standardization of environment services such as Posix and X Windows allow for portability of tools and applications, but it will also permit a rapid growth in the development and use of reusable software components. The goal of a common environment base -- such as the Common APSE Interface Set{45} and Portable Common Tool Environment{46} -- seems almost in reach, and commercial versions of these environment databases are already available.
The primary barrier to developing reliable software is the difficulty involved in gaining insight into the software so that it can be reused effectively. It is my belief that software engineering environments succeed to the degree that they provide insight by analysis tools, information structures, flexible user interface systems, and documentation systems. The environments of today often work in spite of their designer's goals or work poorly.
Future environments will obviously exploit the exponential growth in computational power made available by hardware advances and will likewise seek to exploit the sudden availability of network connectivity. Perhaps not so obvious is the need to break a "complexity barrier"{47} to help users make effective use of these systems. It is exactly this help that software engineering environments must give.
Environmental structure can be characterized by answers to the following questions about its components. For the purposes of this article hardware is generally excluded.
One interesting approach to environment support is based on the notion of computation media. These are more or less "software spreadsheets" in which programming is both textless and naively understandable. One example of this approach is Boxer.{48}
People have a lot of common-sense knowledge about physical space that can be used to make computers more comprehensible. The spatial metaphor encourages people to interpret the organization of the computational system in terms of spatial relationships. Using a Boxer system is like moving around in a large two-dimensional space. All computational objects are represented in terms of boxes, which are regions on the screen that contain text, graphics, or other boxes. Naive realism is an extension of the "what you see is what you have" idea that has become commonplace in the design of text editors and spreadsheets but, unfortunately, not for programming languages. The point is that users should be able to pretend that what they see on the screen is their computational world in its entirety.{48}
Boxer, along with similar systems such as HyperCard, provides a dynamic computation system in which the user alternates between the view of programmer and user, creating, extending, and customizing computational structures. In a real sense it can be viewed as a textless return to InterLisp or Smalltalk.
A more powerful but similar paradigm is that of Hypertext or Hypermedia.{49, 50, 51, 52} Ted Nelson, one of the pioneers of Hypertext, once defined it as "a combination of natural language text with the computer's capacity for interactive branching, or dynamic display -- of a non-linear text -- which cannot be printed conveniently on a conventional page."{50}
A Hypertext system structures information by linking individual documents. A software development process may be viewed as the production of an interrelated series of documents. These documents include operational concept documents, reference manuals, test plans, acceptance test results, review notes, source code, graphical designs, and a myriad of other documents.
The Hypertext system provides a dynamic and flexible means of coordinating this information that takes the place of traditional, inflexible database-centered techniques. The author is designing an environment based on an integrated collection of tools (including an Ada compilation system) that supports the development of Ada software systems based on the Hypertext model. The Hypertext links can be exploited by environment tools as well as software engineers. A configuration management tool, for example, will use links created by the Ada compiler's library manager. This system will be more closely coupled to a compilation facility than other projects, such as that mentioned by Bigelow.{51} -- S.B.