加拿大系统工程教学dissertation Systems Engineering Educati

加拿大系统工程教学dissertation Systems Engineering Education

加拿大系统工程教学dissertation.pdf

Andrew P. Sage, Life Fellow, IEEE
Abstract—We discuss some basic principles underlying systems
engineering, and the translation of these principles to practices
such as to enable the engineering of trustworthy systems of all types
that meet client needs. This special issue is concerned with systems
engineering education. Thus, it is inherently also concerned with
systems engineering, as this provides a major component of the
material that is important for systems engineering education. After
setting forth some of the necessary ingredients for success in systems
engineering, we devote some comments to objectives for and
needs in systems engineering education.
Index Terms—Engineering education, knowledge engineering,
systems engineering.
I. WHAT IS SYSTEMS ENGINEERING?
THE PAPER is concerned with the engineering of systems,
or systems engineering. It is also concerned with the processes
needed to bring about trustworthy systems in an effective
and efficient manner.We are also and especially concerned with
strategic level systems engineering, or systems management,
that is needed to select an appropriate process and ways to provide
technical direction over this process. We begin our effort
by first discussing the need for systems engineering, and then
providing several definitions of systems engineering. We next
present a structure describing the systems-engineering process.
The result of this is a lifecycle model for systems engineering
processes. This is used to motivate discussion of the functional
levels, or considerations, involved in systems engineering efforts:
• systems engineering methods and tools, or technologies
• a systems methodology, or process, as a set of phased activities
that support efforts to engineer the system, and
• systems management.
Fig. 1 illustrates the natural hierarchical relationship among
these levels. Systems engineers are very concerned with each of
these three functional levels. Products (and services) are engineered
through the use of an appropriate process, or processes.
The tailoring of a process for use on a specific instance is accomplished
through systems management. The drivers of systems
 

 

 

management include the external opportunities and pressures,#p#分页标题#e#
and the internal strengths and weaknesses of a given systems engineering
organization, as well as the organizational leadership
and culture associated with the organizations associated with
the tasks at hand. There are a variety of tools and methods, and
technologies needed at the level of product, process, and systems
management. Appropriate measurements are also needed
at all three levels.
Manuscript received December 8, 1988; revised March 15, 1999.
The author is with the Department of Systems Engineering and Operations
Research, George Mason University, Fairfax, VA 22030-4444 USA (e-mail:
[email protected]).
Publisher Item Identifier S 1094-6977(00)04788-X.
Fig. 1. Systems engineering as method, process, and management.
Fig. 2. Systems engineering as a management technology.
Systems engineering is a management technology—which
involves the interactions of science, the organization, and the
environment, and the information and knowledge base that supports
each, as shown in Fig. 2. Technology is the result of, and
represents the totality of, the organization, application, and delivery
of scientific knowledge for the presumed enhancement of
society. This is a functional definition of technology as a fundamentally
human activity. Associated with this definition is the
fact that a technology inherently involves a purposeful human
extension of one or more natural processes. Management involves
the interaction of the organization with the environment.
Consequently, a management technology involves the interaction
of science, the organization, and the environment. Associated
with this must be the information and knowledge that enables
understanding and action to effect change.
The purpose of systems engineering is to support individuals
and organizations that desire improved performance through
technology. This is generally obtained through the definition,
1094–6977/00$10.00 © 2000 IEEE
SAGE: SYSTEMS ENGINEERING EDUCATION 165
Fig. 3. Systems engineering as a broker of knowledge to enable specifications
of system architecture and ultimate engineering solutions at any of six levels.
development, and deployment of technological products, services,
or processes that support functional objectives and which
fulfill needs. Thus, systems engineering is inherently associated
with user organizations and humans in fulfillment of its
objectives. The engineering of systems also involves the interaction
with humans and organizations who are responsible for the
physical implementation of systems. Systems engineers generally
play an important role as brokers of information and knowledge,
and in associated technical direction efforts, in working
both with user enterprises and implementation specialists who#p#分页标题#e#
accomplish the actual realization of physical systems. Fig. 3 illustrates
these conceptual interactions. The using enterprise will
have various functional needs and the functional architecting
and conceptual design part of a systems engineering effort is
concerned with expressing these needs in the form of a functional
architecture. Systems engineers are also concerned with
translation of this functional architecture into a physical architecture
which describes the logical breakdown of the system ultimately
to be constructed in such a way that partitioning of the
systems into various subsystems is then possible. Each of these
subsystems should be as independent as possible and should be
such that integration of them after implementation is as straightforward
and feasible as possible. This physical architecture, or
logical design description of the system, is next translated into
an implementation architecture that provides guidance for various
implementation contractors in bringing about the various
subsystems, which comprise the system. This is also represented
in Fig. 3, which shows the three major architectural perspectives,
or views, of a system.
The products, and associated knowledge, transferred to
the enterprise, or user or client, organization through the
engineering of systems may represent support at three levels:
product, process, or management. Within each of these three
levels, new products, processes, or organizational networks and
scope may be enabled. In a very large number of situations,
there will be the need to integrate these within existing or
legacy systems of products, processes, or management. Thus,
there are six levels of support provided by systems engineering,
and the role of systems integration in these is very strong, as
also suggested in Fig. 3.
We can think of a physical, or more properly stated natural,
science basis for systems engineering, a organizational and social
science basis, and an information science and knowledge
basis. The natural science basis involves primarily matter and
energy processing. The organizational and social science basis
involves human, behavioral, economic, and enterprise concerns.
The information science and knowledge basis is often very difficult
to support effectively. This is so since knowledge is not a
truly fundamental quantity but one that derives from the structure
and organization inherent in the natural sciences, and the
organizational and social sciences. It also results from the purposeful
uses to which information is to be put, and the experiential
familiarity of information holders with the task at hand
and the environment into which the task is imbedded such as
to enable interpretation of information, within an appropriate
context, as knowledge. Thus the presence of information and#p#分页标题#e#
knowledge, as information embedded within context, in Fig. 2
is especially important. This representation stresses the major
ingredients that systems engineers must necessarily deal with
in their approach to the management technology that is systems
engineering: the natural and physical sciences, organizations
and the humans that comprise them, information and knowledge
brokering, and the broad scope environment in which these are
imbedded.
There are several drivers of new technologies. The natural
and physical sciences provide new discoveries that can be
converted into technological innovations. There must be a
marketplace need for technological innovations. Knowledge
perspectives enable the forecasting of the need for innovation.
Innovation results when new knowledge principles are applied
to produce new and different products and services, and
associated knowledge practices, that fulfill a societal need.
There is a need to insure sustainable development and the
intergenerational and intragenerational equity considerations
associated with sustainability. This leads to the notion of
a technical system, an enterprise system, and a knowledge
system. Management, meaning management of the environment
for each of these, is needed. Thus, systems engineers
often act as brokers of knowledge across the enterprises having
needs for support and the various implementation specialists
whose efforts results in detailed construction of innovative
products and services that provide this support. Fig. 4 illustrates
these interrelations. It also indicates that systems engineering
knowledge is comprised of:
1) Knowledge Perspectives—which represent the view that
is held relative to future directions in the technological
area under consideration;
2) Knowledge Principles—which generally represent
formal problem solving approaches to knowledge, generally
employed in new situations and/or unstructured
environments; and
 

 

3) Knowledge Practices—which represent the accumulated
wisdom and experiences that have led to
the development of standard operating policies for
well-structured problems.
These interact together and are associated with learning to
enable continual improvement in performance over time. It is
on the basis of the appropriate use of these knowledge types
166 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 30, NO. 2, MAY 2000
Fig. 4. Systems engineering knowledge and results of its effective use.
that we are able to accomplish the technological system design#p#分页标题#e#
and management system design that leads to a new innovative
product or service.
We continue our discussion and definition of systems engineering
by indicating one possible structural definition. Systems
engineering is management technology to assist and support
policymaking, planning, decision-making, and associated
resource allocation or action deployment. It accomplishes this
by quantitative and qualitative formulation, analysis, and interpretation
of the impacts of action alternatives upon the needs
perspectives, the institutional perspectives, and the value perspectives
of clients to a systems engineering study. The key
words in this definition are formulation, analysis, and interpretation.
In fact, all of systems engineering can be thought
of as consisting of formulation, analysis, and interpretation activities.
We may exercise these in a formal sense, or in an as
if or experientially based intuitive sense. Each of the essential
phases of a systems engineering effort—definition, development,
and deployment—is associated with formulation, analysis,
and interpretation efforts. These enable us to define the
needs for a system, develop the system, and deploy it in an operational
setting and provide for maintenance over time. These
are the components comprising a framework for systems engineering,
as shown in Fig. 5. This framework is comprised of
three phases—definition, development, and deployment—and
three steps within each phase—formulation, analysis, and interpretation.
This is a very aggregated representation of the systems
engineering process. Generally, a more detailed representation
is needed. Fig. 6, for example, represents a five phase representation
of the systems engineering process. This provides a
more realistic view of the efforts needed to engineer a system. It
shows, for example, that one of the major activities of systems
engineering is that of design. It represents the three perspectives,
or views, on design and associated architecting that are taken by
systems engineers:
• preliminary conceptual design and functional architecting;
• logical design or physical system architecting; and
• detailed design or implementation architecting.
A number of questions may be posed with respect to formulation,
analysis, and interpretation that clearly indicate the role
of values in every portion of a systems-engineering effort. Issue
Fig. 5. A systems engineering framework comprised of three phases and three
steps per phase.
 

#p#分页标题#e#
 

Fig. 6. One of several possible life cycle models for systems engineering.
formulation questions of importance in this regard are the following.
• What is the problem? The needs? The constraints? The
alterables?
• How do the client and the analyst bound the issue?
• What objectives are to be fulfilled?
• What alternative options are appropriate?
• How are the alternatives described?
• What alternative state of nature scenarios are relevant to
the issue?
Analysis questions of importance are the following.
• How are pertinent state variables selected?
• How is the issue formulation disaggregated for analysis?
• What generic outcomes or impacts are relevant?
• How are outcomes and impacts described across various
societal sectors?
• How are uncertainties described?
• How are ambiguities and other information imperfections
described?
• How are questions of planning period and planning
horizon dealt with?
Interpretation concerns with respect to value influence are the
following.
• How are values and attributes disaggregated and structured?
SAGE: SYSTEMS ENGINEERING EDUCATION 167
• Does value and attribute structuring and associated formal
elicitation augment or replace experience and intuitive affect?
• How are flawed judgment heuristics and cognitive information
processing biases dealt with?
• Are value perspectives altered by the phase of the systems
engineering effort being undertaken?
Finally, how is total issue resolution time divided between
formulation, analysis, and interpretation? This is important because
the allocation of resources to various systems engineering
activities reflect the value perspectives of the analyst and the
client. These questions associated with formulation, analysis,
and interpretation need to be asked across all of the phases of
systems engineering effort. This has very strong implications
for the practice of systems engineering.
The efforts of some systems engineers may be primarily associated
with the enterprise that ultimately is to become the client
or user of the system to be engineered. They may also be associated
with a systems engineering organization as an independent
broker. Alternately, they may be associated with technical direction
and management of the implementation system detailed design,
production, and maintenance. Fig. 7 illustrates the primary
involvement of these three major stakeholders in the engineering
of a system. Often lifecycles are represented in a “V” fashion
where the “downstroke” activities are associated with decomposition
of the effort into smaller and smaller components, realization#p#分页标题#e#
of the components at the bottom of the downstroke,
and then an “upstroke” effort that is comprised of various integration
efforts to form the complete system. The major efforts
of the enterprise or user group is in conceptualizing the need
for a system. The major efforts of an independent system engineering
organization are in developing physical architectures for
the system, and in taking on configuration control and management
roles relative to implementation of the system. The major
roles of implementation contractors include realization of the
system. These roles are not mutually exclusive and they overlap
over time and across the several phases of activity in engineering
the system, rather than the seemingly abrupt transitions of activity
shown in Fig. 7.
By adopting the management technology of systems engineering
and properly applying it, we become very concerned
with making sure that correct systems are engineered, and not
just that the system is correct according to some potentially
ill-conceived notions of what the system should do. To ensure
that correct systems are engineered requires that considerable
emphasis be placed on the front-end of the systems lifecycle.
It also requires attention to various verification and validation
efforts that ensure that the engineered system satisfies not only
the technological specifications (verification that the system is
correct) but that it performs in a manner such as to satisfy user
needs (validation that it is a correct system) as well.
To support these ends, there needs to be considerable emphasis
on the accurate definition of a system, what it should do,
and how people should interact with it before one is produced
and implemented. In turn, this requires emphasis upon conformance
to system requirements specifications, and the development
of standards to insure compatibility and integratibility of
system products. Such areas as documentation and communi-
Fig. 7. “V” representation of the systems engineering process illustrating
major roles for three primary stakeholders in the engineering of a system.
cation are important in all of this. Thus, we see the need for
the technical direction and management technology efforts that
comprise systems engineering across all phases associated with
engineering a system.
 

 

II. KNOWLEDGE IN SYSTEMS ENGINEERING
Clearly, one form of knowledge leads to another. Knowledge
perspectives may create the incentive for research that leads
to the discovery of new knowledge principles. As knowledge#p#分页标题#e#
principles emerge and are refined, they generally become
imbedded in the form of knowledge practices. Knowledge
practices are generally the major influences of the systems that
can be acquired or fielded. These knowledge types interact
together, as suggested in Fig. 4, which illustrates how these
knowledge types support one another. In a nonexclusive way,
they each support one of the principle lifecycles associated with
systems engineering. Knowledge practices are generally very
much associated with the acquisition, or manufacturing, or production,
of new systems. Knowledge principles are very much
associated with the fundamental knowledge that is needed for
research and development, and knowledge perspectives suggest
systems planning and marketing directions. These knowledge
forms flow naturally from one to the other and Fig. 4 also
illustrates a number of feedback loops that are associated with
learning to enable continual improvement in performance over
time.
The use of the term knowledge is very purposeful here. It
has long been regarded as essential in systems engineering and
management to distinguish between data and information. Information
is generally defined as data that is of value for decision
making. For information to be successfully used in decision
making, it is necessary to associate context and environment
with it. The resulting information, as enhanced by context
and environment, results in knowledge. Appropriate information
management and knowledge management are each necessary
for high quality systems engineering and management.
It is on the basis of the appropriate use of the three knowledge
types depicted in Fig. 3, that we are able to accomplish the
technological system planning and development and the management
system planning and development, that lead to a new
innovative product or service. All three types of knowledge are
needed. The environment associated with this knowledge needs
168 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 30, NO. 2, MAY 2000
to be managed, and this is generally what is intended by use of
the term knowledge management. Also, the learning that results
from these efforts is very much needed, both on an individual
and an organizational basis.
As indicated in [1]–[3], and the references contained therein,
there are three different primary systems engineering lifecycles
for technology growth and change:
• System Planning and Marketing
• Research, Development, Test and Evaluation (RDT&E),
and
• System Acquisition, Production, or Procurement.
These are each generally needed, and each primarily involves
use of one of the three types of knowledge. There are a number
of needed interactions across these lifecycles for one particular#p#分页标题#e#
realization of a system acquisition lifecycle. It is important that
efforts across these three major systems engineering lifecycles
be integrated. There are many illustrations of efforts that were
dramatically successful efforts in RDT&E, but where the overall
results represent failure because of lack of consideration of planning,
or of ultimate manufacturing needs of a product while it
is in RDT&E.
In our definition of systems engineering, we indicated that
systems engineers are concerned with the appropriate
• definition,
• development, and
• deployment of systems.
These comprise a set of phases for a systems engineering lifecycle,
as illustrated in Fig. 5. There are many ways to characterize
the lifecycle phases of systems engineering processes, and
a considerable number of them are described in [1]–[3]. Each of
the lifecycle models, and those with are outgrowths of them, are
comprised of these three phases. For pragmatic reasons, a typical
lifecycle will contain more than three phases, as we shall
soon indicate.
III. THE IMPORTANCE OF TECHNICAL DIRECTION AND
SYSTEMS MANAGEMENT
In order to resolve large scale and complex problems, or to
manage large systems of humans and machines, we must be
able to deal with important contemporary issues that involve and
require:
1) many considerations and interrelations;
2) many different and perhaps controversial value judgments;
3) knowledge from several disciplines;
4) knowledge at the levels of principles, practices, and perspectives;
5) considerations involving definition, development, and
deployment of systems;
6) considerations that cut across the three different lifecycles
associated with systems planning and marketing,
RDT&E, and system acquisition or production;
7) risks and uncertainties involving future events which are
difficult to predict;
8) a fragmented decision making structure;
9) human and organizational need and value perspectives,
as well as technology perspectives; and
10) resolution of issues at the level of institutions and values
as well as the level of symptoms.
Those involved with the professional practice of systems engineering
must use of a variety of formulation, analysis, and
interpretation aids for evolution of technological systems and
management systems. Clients and system developers alike need
this support to enable them to cope with multifarious large-scale
issues. This support must avoid several potential pitfalls. These
include the following 12 deadly systems engineering transgressions.
1) There is an over-reliance upon a specific analytical
method or tool, or a specific technology, that is advocated
by a particular group.
2) There is a consideration of perceived problems and issues#p#分页标题#e#
only at the level of symptoms, and the development
and deployment of “solutions” that only address symptoms.
3) There is a failure to develop and apply appropriate
methodologies for issue resolution that will allow identification
of major pertinent issue formulation elements,
a fully robust analysis of the variety of impacts on stakeholders
and the associated interactions among steps of
the problem solution procedure, and an interpretation
of these impacts in terms of institutional and value
considerations.
4) There is a failure to involve the client, to the extent necessary,
in the development of problem resolution alternatives
and systemic aids to problem resolution.
5) There is a failure to consider the effects of cognitive biases
that result from poor information processing heuristics.
6) There is a failure to identify a sufficiently robust set of
options, or alternative courses of action.
7) There is a failure to make and properly utilize reactive,
interactive, and proactive measurements to guide
the systems engineering efforts.
 

 

8) There is a failure to identify risks associated with the
costs and benefits, or effectiveness, of the system to be
acquired, produced, or otherwise fielded.
9) There is a failure to properly relate the system that is
designed and implemented with the cognitive style and
behavioral constraints that effect the user of the system,
and an associate failure of not properly designing the
system for effective user interaction.
10) There is a failure to consider the implications of strategies
adopted in one of the three lifecycles (RDT&E, acquisition
and production, and planning and marketing)
on the other two lifecycles.
11) There is a failure to address quality issues in a comprehensive
manner throughout all phases of the lifecycle,
especially in terms of reliability, availability, and maintainability.
12) There is a failure to properly integrate a new system together
with heritage or legacy systems that already exist
and which the new system should support.
Systems engineers take on technical roles associated with
the engineering of systems. They take on management roles
associated both with identification of appropriate processes and
SAGE: SYSTEMS ENGINEERING EDUCATION 169
with technical direction of implementation efforts and associated
overall configuration control. They take on roles associated with
management of the environment surrounding the engineering of
systems. Failures can occur in any of these. Most of the failures#p#分页标题#e#
are generally associated with systems management failures.
In general, we may approach issues from an inactive, reactive,
interactive, or proactive perspective.
• Inactive—This denotes an organization that does not
worry about issues and which does not take efforts to resolve
them. It is a very hopeful perspective, but generally
one that will lead to issues becoming serious problems.
• Reactive—This denotes an organization that will examine
a potential issue only after it has developed into a real
problem. It will perform an outcomes assessment and after
it has detected a problem, or failure, will diagnose the
cause of the problem and, often, will get rid of the symptoms
that produce the problem.
• Interactive—This denotes an organization that will attempt
to examine issues while they are in the process of
evolution such as to detect them at the earliest possible
time. Issues that may cause difficulties will not only be
detected, but efforts at diagnosis and correction will be
implemented as soon as they have been detected. This will
involve detect of problems as soon as they occur, diagnose
of their causes, and correction of difficulty through recycling,
feedback and retrofit to and through that portion of
the lifecycle process in which the problem occurred. Thus,
the term interactive is, indeed, very appropriate.
• Proactive—This denotes an organization that predicts the
potential for debilitating issues and which will synthesize
an appropriate lifecycle process that is sufficiently mature
such that the potential for issues developing is as small as
possible.
It should be noted that there is much to be gained by a focus on
process improvements in efforts from any of the last three perspectives.
While proactive and interactive efforts are associated
with greater capability and process maturity that are reactive efforts,
reactive efforts are still needed [2]. Inactivity is associated
with failure, in most cases.
Management of systems engineering processes, which we
call systems management, is very necessary for success. There
are many evidences of systems engineering failures at the level
of systems management. Often, one result of these failures is
that the purpose, function, and structure of a new system are
not identified sufficiently before the system is defined, developed,
and deployed. These failures generally cause costly mistakes
that could truly have been avoided. Invariably this occurs
because either the formulation, the analysis, or the interpretation
efforts (or all of them perhaps) are deficient. A major objective
of systems engineering, at the strategic level of systems
management, is to take proactive measures to avoid these difficulties.#p#分页标题#e#
Contemporary efforts in systems engineering contain
a focus on: tools and methods, and technologies for the engineering
of systems, and associated metrics; systems methodology
for the lifecycle process of definition, development and
deployment that enables appropriate use of these tools, methods,
and technologies; and systems management approaches that enables
the proper imbedding of systems engineering product and
process evolution approaches within organizations and environments.
In this way, systems engineering and management provides
very necessary support to the role of conventional and
classical engineering endeavors through the implementation of
various physical systems architectures as useful technological
products. Fig. 1 attempts to show this conceptual model of systems
engineering.
System management and integration issues are of major
importance in determining the effectiveness, efficiency, and
overall functionality of system designs. To achieve a high
measure of functionality, it must be possible for a system,
meaning a product or a service, to be efficiently and effectively
produced, used, maintained, retrofitted, and modified
throughout all phases of a lifecycle. This lifecycle begins with
need conceptualization and identification, through specification
of system requirements and architectures, to ultimate system
installation, operational implementation or deployment, evaluation,
and maintenance throughout a productive lifetime. It
is important to note that a system, product or service, that is
produced by one organization may well be used as a process,
or to support a process, by another organization.
Virtually all studies of the engineering of systems show
that the major problems associated with the production of
trustworthy systems have more do with the organization and
management of complexity than with direct technological
concerns that affect individual subsystems and specific physical
science areas. Often the major concern should be more associated
with the definition, development, and use of an appropriate
process, or product line, for production of a product than it is
with the actual product itself, in the sense that direct attention
to the product or service without appropriate attention to the
process leads to the fielding of a low quality and expensive
product or service.
IV. LIFECYCLE METHODOLOGIES, OR PROCESSES, FOR
SYSTEMS ENGINEERING
As we have noted, systems engineering is the creative process
through which products, services, or systems that are presumed
to be responsive to client needs and requirements are conceptualized
or specified or defined, and ultimately developed and
deployed. There are at least twelve primary assertions implied
by this not uncommon definition of systems engineering, and#p#分页标题#e#
they apply to the development of software intensive systems, as
well as to hardware and physical systems.
1) Systems planning and marketing is the first strategic
level effort in systems engineering. It results in the determination
of whether or not a given organization should
undertake the engineering of a given product or service.
It also results in a, at least preliminary, determination of
the amount of effort to be devoted to RDT&E and the
amount to actual system acquisition or production.
2) Creation of an appropriate process or product line for
RDT&E and one for acquisition is one result of system
planning and marketing. The initial systems planning
and marketing efforts determine the extent to which
RDT&E is a need, and also determine the acquisition
process characteristics that are most appropriate.
170 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 30, NO. 2, MAY 2000
3) An appropriate planning process leads to efficient and
effective RDT&E, and to the actual system acquisition
which follows appropriate RDT&E.
4) The first phase of any systems engineering lifecycle effort
results in the identification or definition of specifications
for the product or service that is to result from
the process.
5) Systems engineering is a creative process based effort.
6) Systems engineering activities are conceptual in nature
at the initial phases of effort, for either of the three
generic lifecycles, and become operational in later
phases.
7) A successful systems engineering product or service
must be of high quality and responsive to client needs
and requirements.
8) A successful systems engineering product, or service,
generally results only from a successful systems engineering
process.
9) An appropriate systems engineering process is, generally,
the result of successful systems management, and
appropriate planning and marketing.
10) Appropriate systems engineering efforts need necessarily
be associated with systematic measurements to
insure high quality information as a basis for decision
making across the three generic systems engineering
lifecycles.
11) Appropriate systems engineering efforts are necessarily
attuned to organizational and environmental realities as
they affect both the client organization and the systems
engineering organization.
12) Systems engineering efforts are, of necessity interactive.
However, they transcend interactivity to include proactivity.
Good systems engineering practice requires that the systems
engineer be responsive to each of these twelve ingredients for
quality effort. Clearly, not all members of a systems engineering
team are responsible for, and participate in, each and every systems#p#分页标题#e#
engineering activity.
V. EVOLUTION OF ENGINEERING AND ENGINEERING
EDUCATION
Many conventional definitions of engineering suggest that it
is the application of scientific principles to the optimal conversion
of natural resources into products and systems for the
benefit of humankind. The notion that engineering is concerned
with effective and efficient use of resources for the betterment
of humankind is certainly correct. There are many constraints
affecting this use and engineering is much concerned with developing
solutions under constraints. Initially, these resources
were considered to be natural resources. Today, they are considered
to be any of the four major resources or capital, as unspent
resources are often now called:
• natural resources, or natural capital;
• human resources, or human capital;
• financial resources, or financial capital; and
• information and knowledge resources, or information and
knowledge capital.
This enlarged concept of resources enables us to include such
important contemporary knowledge intensive efforts as biotechnology
and biomedical engineering. Science, on the other hand,
is primarily concerned with the discovery of new knowledge.
There is no inherent notion of purpose in scientific discoveries,
although obviously many scientific investigations are directed at
knowledge that will be of ultimate beneficial use to humanity.
Much of the world has been transformed by technology, as
evidenced in an excellent work [4] that describes the history
of American invention and innovation over the century from
1870–1970. While this period of time could hardly be called
the information age, Beniger [5] indicates that it was actually
during this period that the essence of the contemporary information
age began in America. Microelectronics and integrated
circuit related efforts, including digital computers and communications,
became the “glamour” technologies of the 1970s and
1980s. These technologies have produced profound impacts on
society and on the engineering profession. The ease of development
and the power of integrated circuits have actually changed
the implementation architecture for electrical circuits and the
performance characteristics of the resulting systems. This has
led engineers to actively search for digital solutions to problems
that are not themselves inherently digital. For example, the
simulation of continuous time dynamic physical systems, such
as aircraft, is now accomplished almost totally digitally, even
though the physical systems themselves are continuous time
systems for which much analog computer technology had been
developed in the 1950s and 60s. This “digital everything” trend
has resulted from the major developments in semiconductors,#p#分页标题#e#
abilities at very large scale integration of electronic circuits, the
resulting microprocessor based systems, and associated major
reductions in size and cost of digital computer components and
systems.
The digital revolution [6] has led to a death of distance [7], the
merging of telecommunications technology and computer technology
into information technology, and networked individuals
and organizations. Major characteristics of this change include
great speed [8]. More importantly, they enable networking and
communications. They also result in the major necessity for all
of engineering, especially systems engineering, to be especially
concerned with social choice and value conflicts issues [9] that
surround strategic management of the intellectual capital [10] as
a major new form of capital resource that has been in very large
part brought about by the information technology revolution and
the use of information technology for organizational and societal
improvement. We have seen the initial focus on data in the
early days of computers shift to a focus on information and information
technology in the decade of the 1990s. Now we sense
the imbedding of information concerns into greater concerns
that affect knowledge resources and knowledge management.
Transdiciplinary issues of knowledge integration need to be addressed
well if we are able to address the concerns of the early
part of the 21st Century. There are many influences of these
innovative changes [11] [12]. These are bringing about major
changes [13] and needs for engineering education to adapt programs
to these changes such that the customers of engineering
education, students and employers, remain satisfied with educational
product quality.
 

 

SAGE: SYSTEMS ENGINEERING EDUCATION 171
Comments on the changing environment for engineering and
engineering education are commonplace and issues such as the
following are often cited [14].
1) Availability of a many new engineered materials, and an
associated much larger “design space” from which the
engineer must choose.
2) Pervasive use of information technology in the products
and processes of engineering.
3) Increasing number and complexity of constraints on acceptable
engineering solutions. Where cost and functionality
were once the dominant concerns, ecological and
natural resource concerns, sustainability, safety, and reliability
and maintainability are now also major concerns.
4) Globalization of industry and the associated shift from
a nationally differentiated engineering enterprise to one#p#分页标题#e#
that is far more global.
5) Major increases in the technical depth needed in manufacturing
and service sectors, both in terms of absolute
specific technical knowledge and the breadth of knowledge
needed.
6) Expanded role of the engineer as part of integrated
product and process teams, and the broad business
knowledge required.
7) Increased pace of change in which there appears to be less
time to assimilate and adapt.
Each of these, individually and particularly in combination,
lead to many new challenges for engineering education, especially
as they relate to the technical direction and knowledge
brokerage needed to bring about trustworthy systems through
systems engineering.
In one notable and particularly relevant work [15], relevant,
attractive, and connected engineering education is outlined as
education that results from engineering programs that undertake
several important action items.
1) Establish Individual Missions for Engineering Colleges,
such that an effective planning process that enacts
a clear vision supportive of excellence drives each
program.
2) Re-Examine Faculty Rewards, such as to identify incentives
that assure commitment and which support the
programmatic mission.
3) Reshape the Curriculum to enable relevance, attractiveness,
and connectivity.
4) Ensure Lifelong Learning of all, supported in part by
new and innovative technologies for education [16].
5) Broaden Educational Responsibility; such that engineering
programs provide support for elementary and secondary
education.
6) Accomplish Personnel Exchanges, such that faculty are
able to obtain relevant experience in industry and government,
and such that industry and government experience
are able to contribute their talents to programs in engineering
education.
7) Establish Across the Campus Outreach, such that high
quality and relevant courses in engineering are made
available throughout the university.
8) Encourage Research/Resource Sharing, Open Competition
Based on Peer Review, and Enhanced Technology
Transfer.
The attributes associated with reshaping the curriculum are
of special importance in that these are directly focused on educating
students for careers as professional engineers, for research,
for planning and marketing, and for the many other functions
performed by engineers. The major ingredients associated
with reshaping the curriculum were suggested as:
• team skills, and collaborative, active learning;
• communication skills;
• a systems perspective;
• an understanding and appreciation of diversity;
• appreciation of different cultures and business practices,
and understanding that engineering practice is now global;
• integration of knowledge throughout the curriculum a#p#分页标题#e#
multidisciplinary perspective;
• commitment to quality, timeliness, continuous improvement;
• undergraduate research and engineering work experience;
• understanding of social, economic, and environmental impact
of engineering decisions;
• ethics.
 

 

Each of these is particularly important for engineering education,
and especially for systems engineering education. This is
especially so in light of relevant works that examine the role of
technology and values in contemporary society [17] and which
stress the need for engineering to become more integrated with
societal and humanistic concerns, such as to enable engineers to
better cope with issues and questions of economic growth and
development, and sustainability and the environment [18].
VI. OBJECTIVES FOR SYSTEMS ENGINEERING EDUCATION
Engineering education is a professional activity and an intellectual
activity. It is necessary that the faculty responsible for
this educational delivery in engineering remain at the cutting
edge of relevant technologies, including emerging technologies,
as technology does change rapidly over time. Research is, therefore,
an absolute essential in engineering education. It is possible
through relevant research, and associated knowledge principles,
to develop new engineering knowledge principles and
practices that are relevant to societal improvements that result
from better use of information and technological innovations.
Research is exceptionally important for engineering education,
as it is strongly supportive of the primary educational objective
of the university. It is vital to remain vigilant relative to
the educational mission, and this requires that faculty remain
at the cutting edge of technology in order that they are able to
provide education, meaning teaching, at that forefront. It is because
of the need to remain current in the classroom in order to
deliver education for professional practice that the strong need
and a mandate for faculty research in engineering necessarily
emerges.
This suggests that research activities in engineering education
should generally be very student oriented. It suggests that students
are an inseparable and integral part of faculty research. It
172 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 30, NO. 2, MAY 2000
suggests a major role for students in development and cooperative/
internship ventures with industry and government. This creates
the strong need for sponsored research and internships that#p#分页标题#e#
assure the needed industry- government-university interactions.
In addition to being intimately associated with the educational
process, sponsored research also provides faculty with released
time from exclusively teaching efforts for scholarly pursuits
necessary to retain currency in the classroom. Also needed are
innovative efforts to transfer research in emerging technologies
with potential marketplace success to a position where these results
are useful in system acquisition. To bring this about satisfactorily
requires much attention to risk management and the
necessary determination of the intersections where marketing,
RTD&E and acquisition can each enjoy success.
The knowledge and skills required in engineering, and in engineering
education, come from all of the sciences, and from
the world of professional practice. This suggests that faculty
in a professional school of engineering need to keep abreast
of progress in relevant sciences, both the natural sciences and
the economic and social sciences, and the mathematical and engineering
sciences. Taken together, these comprise knowledge
principles. It suggests also that engineering educators must keep
abreast of and contribute to industrial practices in relevant professional
practice areas. It is for this reason that engineering
schools are and must remain professional schools. This is also
why close industry-university and government–university interactions,
becomes a most desirable, and in fact essential, part
of successful, high-quality engineering education programs.
Efforts in engineering must necessarily involve likely future
technological developments as well, if the customers for systems
engineering education are to be satisfied. Thus, we see
the need for knowledge practices, knowledge principles, and
knowledge perspectives in engineering education. These knowledge
components, and the necessary learning to enable transition
and natural evolution of one form of knowledge into the
other, are very important for both technology transfer and for
engineering education as they relate to engineering in general
and systems engineering in particular.
A number of issues relative to engineering education are discussed
in [19] and the references therein. One of the major new
developments in engineering education is Engineering Criteria
2000 [20], which is comprised of criteria intended to emphasize
quality and preparation for professional practice. The criteria
retains the traditional core of engineering, math, and science requirements.
However, they also place importance on formal efforts
that stress teamwork, communications, and collaboration
as well as global, economic, social, and environmental awareness.
They are based on the premises that:
• technology has been a driver of many of the changes occurring#p#分页标题#e#
in society over the last several years;
• it will take on an even larger role in the future;
• the engineering education accreditation process must promote
innovation and continuous improvement to enable
institutions to prepare professional engineers for exciting
future opportunities.
These criteria are focused on insuring competence, commitment,
communications, collaboration, and the courage needed
for individual responsibility. These, augmenting the usual listing
of competence and assumption of individual responsibility as
the two traditionally accepted key characteristics of a professional,
might be accepted as the new augmented attributes of a
mature professional. They should truly support the definition,
development, and deployment of relevant, attractive and connected
(quality) engineering education that will:
• include the necessary foundations for knowledge principles,
practices, and perspectives;
• integrate these fundamentals well through meaningful design,
problem solving, and decision-making efforts;
• be sufficiently practice oriented to prepare students for
entry into professional practice;
• emphasize teamwork and communications, as well as individual
efforts;
• incorporate social, cultural, ethical, and equity issues, and
a sense of economic and organizational realities—and a
sense of globalization of engineering efforts;
• instill an appreciation of the values of personal responsibility
for individual and group stewardship of the natural,
techno-economic, and cultural environment.
• instill a knowledge of how to learn, and a desire to learn,
and to adapt to changing societal needs over a successful
professional career.
The unprecedented technological advances in the information
technologies of computation, communication, software,
and networking create numerous opportunities for enhancing:
our life quality, the quality of such critical societal services such
as health and education, and the productivity and effectiveness
of organizations. We are witnesses to the emergence of new
human activities that demand new processes and management
strategies for the engineering of systems. The major need is
for appropriate management of people, organizations, and
technology as a social system. Systems engineering is basically
concerned with finding integrated solutions to issues that are of
large scale and scope. Educational programs in systems engineering
need to be especially concerned with the emergence of
systems engineers who can cope with these challenges. They
need especially to be concerned with: the three levels of support
for systems engineering efforts—methods, tools, technologies,
and metrics; processes, and systems management. They need#p#分页标题#e#
to be especially concerned with the evolution of technological
innovations through life cycle processes that involve: research,
development, test, and evaluation, planning and marketing; and
systems acquisition, procurement, and manufacturing. They
need to be concerned with efforts that are reactive to observed
deficiencies, interactive to avoid errors to the extent possible;
and proactive, such as to enable the determination of processes
and systems management procedures based on realistic future
perspectives. They must pay critical attention to integration at
the level of product, processes, and systems management; and
they must be aware of the need for knowledge integration itself.
Also, there is much need to be concerned with the knowledge
brokerage and technical directions necessary to insure success
in the engineering of trustworthy and useful large-scale systems
of humans, organizations, and technologies.
Much more could be said and has been said relative to these
important issues as they effect engineering education in general
and systems engineering education in particular [21]–[29]. The
SAGE: SYSTEMS ENGINEERING EDUCATION 173
recent Electronic Industries Association Interim Standard 731,
Systems Engineering Capability Model (SECM) [30] identifies
19 focus areas for systems engineering that fall into three
natural groupings, or categories: Technical, Management, and
Environment. These three groupings correspond closely to the
notions of product, process, and systems management described
here and in [1]–[3], and elsewhere. The technical focus areas
support practices which are indicative of the technical aspects
of systems engineering. They generally correspond well with
definitions and practices contained in two important standards
for Systems Engineering, EIA 632, Processes for Engineering a
System [31], and IEEE STD 1220, Trial-Use Standard for Application
and Management of the Systems Engineering Process
[32]. The systems management focus area practices support the
technical focus areas through planning, control and information
management. These are attempted to incorporate and practices
from systems engineering standards with industry-wide best
practices such as to enhance cost-effectiveness in the engineering
of systems, or systems engineering. The environment
focus areas in the SECM standard represent those practices that
facilitate sustained execution of systems engineering processes
throughout the systems engineering organization. These are
intended to ensure alignment of process and technology development
with systems engineering business objectives. These
practices support the technology and management focus areas.
Fig. 8 represents these 19 focus areas. These comprise a very
useful set of needed abilities for systems engineers. A major#p#分页标题#e#
goal of systems engineering education should be to provide
relevant courses and laboratories that support the attainment of
abilities relative to each of these focus areas. There are a variety
of tools and methods that support satisfactory performance
in each of these focus areas and provision of support from
these will allow for incorporation of the what to do delineated
so well in the standard with the how to do it that is also
needed for trustworthy systems engineering. This is a never
ending continuous improvement effort, as also represented in
Fig. 8. In a similar way, systems engineering education is a
never ending lifelong process that cuts across the three major
dimensions for systems engineering effort shown in Fig. 9
and which includes appropriate lifecycle processes phases and
steps within each.
VII. SUMMARY
We have presented a wide scope discussion of systems engineering
education. We have discussed the emergence of concerns
for large systems of humans, organizations, and technologies.
We have discusses some of the principles of systems engineering
that need necessarily be incorporated into relevant
curricula. We have stressed systems engineering education as
preparation for professional practice as well as for the development
of knowledge principles through research. We have focused
on contemporary concerns relative to educational quality
and responses to these, and educational needs and accreditation
standards for the 21st Century to achieve this quality. A flow
chart of interactions of systems engineering education would
show a very large number of linkages across many related elements
thereby indicating that engineering education itself is a
Fig. 8. Systems engineering capability model focus areas.
Fig. 9. One of the many possible 3-D frameworks for systems engineering.
system of large scale and scope. Our discussion is necessarily
wide scope in that systems engineering education itself is necessarily
wide scope.
A systems engineer must surely understand the principles of
the natural and mathematical sciences. They must have this understanding
in order knowhowto use these to support the definition,
development, and deployment of cost effective and trustworthy
systems and also to have the background necessary to
retain intellectual currency throughout a lifetime of continued
learning. The purpose behind the engineering of systems is the
development of products, services, and processes that are successful
in the marketplace through fulfillment of societal needs.
Technological, organizational, and societal change are the order
of the day, just as they have been throughout history. If these
changes are to be truly effective and effective, over the long
term especially, they must serve societal needs. This suggests#p#分页标题#e#
that change needs necessarily to be guided by principles of social
equity and justice, as well as by concerns for sustainable
development and marketplace competition. There is strong evidence
that this needed guidance does not always occur and that
the hoped for productivity gains from technological advances
may be elusive [33]–[36]. This provides the mandate for a major
component of the social and behavioral sciences, and the political
and policy sciences, in systems engineering education and
in engineering practice as necessary ingredients for success. It
174 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 30, NO. 2, MAY 2000
also provides a mandate for major integrative knowledge components
in systems engineering education and for educational
accreditation standards that reflect these needs, as recognized
in the reengineering efforts for education and engineering education
suggested by a large number of the sources cited here.
While many of these are personal references, there are a vast
number of references to the excellent work of many others contained
therein. These support the emergence of a multidimensional
framework for systems engineering and systems engineering
education, some of the many components of which have
been discussed here.
REFERENCES
[1] A. P. Sage, Systems Engineering. New York: Wiley, 1992.
[2] , Systems Management for Information Technology and Software
Engineering. New York: Wiley, 1995.
[3] A. P. Sage and W. B. Rouse, Eds., Handbook of Systems Engineering
and Management. New York: Wiley, 1999.
[4] T. P. Hughes, American Genesis: A Century of Invention and Technological
Enthusiasm. New York: Viking Penguin, 1989.
[5] J. R. Beniger, The Control Revolution: Technological and Economic Origins
of the Information Society. Cambridge, MA: Harvard Univ. Press,
1986.
[6] D. Tapscott, Growing up Digitally: The Rise of The Net Generation.
New York: McGraw Hill, 1998.
[7] F. Cairncross, The Death of Distance: How the Communications Revolution
Will Change Our Lives. Boston, MA: Harvard Business School
Press, 1997.
[8] S. Davis and C. Myer, Blur: The Speed of Change in the Connected
Economy. Reading, MA: Addison Wesley, 1998.
[9] R. Kling, Ed., Computerization and Controversy: Value Conflicts and
Social Choices. San Diego, CA: Academic Press, 1996.
[10] D. A. Klein, The Strategic Management of Intellectual Capital.
Woburn, MA: Butterworth-Heinemann, 1998.
[11] T. P. Hughes and J. R. Sheehan, “What has influenced computer innovation,”
IEEE Computer, vol. 31, no. 2, pp. 33–43, Feb. 1999.
[12] N. Valéry, “Industry gets religion,” The Economist, Feb. 20, 1999.
[13] M. Detrouzos, What Will Be: How the New World of Information will#p#分页标题#e#
Change our Lives. New York: Harper Collins, 1997.
[14] W. A.Wulf, “The changing nature of engineering,” The Bridge, vol. 27,
no. 2, Summer 1997.
[15] Engineering Education for a Changing World: American Society for
Engineering Education, 1994.
[16] A. Bourne, A. Broderson, and M. Dawant, Eds., The Influence of Information
Technology on Engineering Education. Boca Raton, FL: CRC,
1995.
[17] L. A. Zadeh, “UC-Berkeley computer science commencement address,”
IEEE Trans. Syst., Man, Cybern., vol. 28A, no. 1, pp. 7–8, Feb. 1998.
[18] G. Bugliarrello, “Engineering at the crossroads of our species,” The
Bridge, vol. 28, no. 1, pp. 9–15, Spring 1998.
[19] A. P. Sage, “Electrical engineering education,” in Encyclopedia of
Electrical and Electronics Engineering, J. G.Webster, Ed. New York:
Wiley, 1999, vol. 6, pp. 241–254.
[20] Engineering Criteria 2000: Accreditation Board for Engineering and
Technology, Jan. 1998.
[21] A. P. Sage, “Systems engineering education,” Syst., Man, Cybern. Rev.,
vol. 7, no. 1, pp. 6–9, Feb. 1978.
[22] , “The role of systems engineering in electrical engineering education,”
IEEE Trans. Educ., vol. 22, no. 2, pp. 81–85, May 1979.
[23] , “From philosophical perspectives to practice in the design of
program planning linkages for systems engineering education,” IEEE
Trans. Syst., Man, Cybern., vol. SMC-10, no. 11, pp. 693–695, Nov.
1980.
[24] , “Desiderata for systems engineering education,” IEEE Trans.
Syst., Man, Cybern., vol. SMC-10, no. 12, pp. 777–780, Dec. 1980.
[25] , “Knowledge transfer: An innovative role for information engineering
education,” IEEE Trans. Syst., Man, Cybern., vol. SMC-17, no.
5, pp. 725–728, Sept./Oct. 1987.
[26] J. D. Palmer and A. P. Sage, “Information technology management of
university education—Part I: An overview of university executive functions,”
Int. J. Technol. Manag., vol. 2, no. 3, pp. 391–404, 1987.
[27] , “Information technology management of university education—
Part II: Case histories of information management,” Int. J.
Technol. Manag., vol. 4, no. 1, pp. 77–94, 1989.
[28] , “Information technology management of university education:
Part III—A methodology for information system design and development,”
Int. J. Technol. Manag., vol. 5, no. 2, pp. 217–237, 1990.
[29] A. P. Sage, “Systems engineering and information technology: Catalysts
for total quality in industry and education,” IEEE Trans. Syst., Man,
Cybern., vol. 22, no. 5, pp. 833–864, Sept. 1992.
[30] Systems Engineering Capability Model Interim Standard, EAI/IS 731,
1999.#p#分页标题#e#
[31] Processes for Engineering a System, EIA 632, 1999.
[32] Trial-Use Standard for Application and Management of the Systems Engineering
Process, IEEE STD 1220, 1999.
[33] R. H. McGuckin and K. J. Stiroh, “Computers can accelerate productivity
growth,” Issues in Science and Technology, vol. 14, no. 4, pp.
41–48, Summer 1998.
[34] S. R. Roach, “No productivity boom for workers,” Issues in Science and
Technology, vol. 14, no. 4, pp. 49–56, Summer 1998.
[35] M. M. Moll, Ed., Tech High: Globalization and the Future of Canadian
Education. Ottawa, Canada: Fernwood Publishers, 1997.
[36] U.S. White House Report, “Information technology for the twenty-first
century: A bold investment in America’s future,”, Jan. 24, 1999.
Andrew P. Sage (S’56–M’57–SM’65–F’76–LF’97)
received the B.S.E.E. degree from The Citadel,
Charleston, SC, the S.M.E.E. degree from MIT,
Cambridge, MA, and the Ph.D. degree from Purdue
University,West Lafayette, IN, the latter in 1960. He
received honorary Doctor of Engineering degrees
from the University of Waterloo, Waterloo, ON,
Canada, in 1987 and from Dalhousie University,
Halifax, NS, Canada, in 1997.
He has been a faculty member at the University
of Arizona, University of Florida, and Southern
Methodist University. During 1974–1984, he was Lawrence R. Quarles
Professor of engineering science and systems engineering at the University of
Virginia. During portions of this time, he was Associate Dean for Graduate
Studies and Research, Chair of the Chemical Engineering Department, and
Chair of the Systems Engineering Department. In 1984, he became First American
Bank Professor of Information Technology and Engineering at George
Mason University and the first Dean of the School of Information Technology
and Engineering. In May 1996, he was elected as Founding Dean Emeritus of
the School and also was appointed a University Professor. His initial research
concerned optimization, estimation, system modeling and identification, and
communications and control system design. His current interests include
systems engineering and management efforts in a variety of application areas
including systems integration and reengineering, software systems engineering,
total quality management, cost and effectiveness assessment, and industrial
ecology and sustainable development. He is the author or co-author of a number
of papers in these areas as well as a number of texts that have resulted from his
principal effort, which is teaching in systems engineering and management.
Dr. Sage is an elected Fellow of the American Association for the Advancement
of Science, and the International Council on Systems Engineering (INCOSE).#p#分页标题#e#
He received the Frederick Emmonds Terman Award from the American
Society for Engineering Education, and an Outstanding Service Award from
the International Federation of Automatic Control. He received the first Norbert
Wiener Award as well as the first Joseph G. Wohl Outstanding Career Award
from the IEEE Systems, Man, and Cybernetics Society. In 1994, he received
the Donald G. Fink Prize from the IEEE, and a Superior Public Service Award,
for his service on the CNA Corporation Board of Trustees from the U.S. Secretary
of the Navy. He was Editor of the IEEE TRANSACTIONS ON SYSTEMS MAN
AND CYBERNETICS for the 27 year period 1972 through 1998, and was an Editor
of the IFAC Journal Automatica for the 16 year period 1981 through 1996. He
is currently editor of the John Wiley textbook series on Systems Engineering,
and the INCOSE Wiley journal Systems Engineering. He is also active in other
scholarly editorial efforts. He was President of the IEEE Systems, Man, and Cybernetics
Society for the two-year period 1984–1985. In 2000, he received the
Simon Ramo Medal from the IEEE as well as a Third Millennium Medal.