Advanced Automation for Space Missions/Chapter 2

 

CHAPTER 2

TERRESTRIAL APPLICATIONS: AN INTELLIGENT EARTH-SENSING INFORMATION SYSTEM

2.1 Introduction

Figure 2.1.- Terrestrial applications: An intelligent Earth-sensing information system. PRECEDING PAGE BLANK NOT FILMO*

The Terrestrial Applications Team was charged with identifying a sample near-Earth NASA mission that could be implemented during the next two or three decades and that would require intensive application of artificial intelligence and robotics technologies. The team initially considered a long list of missions that included the design and automated fabrication of a satellite solar power station, weather sensing and prediction, crop assessment, large communication satellites, and disaster monitoring. As the catalogue of possible tasks evolved, it became clear that artificial intelligence would be most useful when applied to missions that generate data at very high rates - such as the NASA applications (Landsat) satellites which provide imaging data of the Earth. The team focused on the development of an integrated, user-oriented, Earth-sensing information system (fig. 2.1) incorporating a maximum of artificial intelligence capability for two primary reasons.

First, substantial economic benefits may accrue from the effective use of an integrated, intelligent remote Earth- sensing system. For example, a reduction in weather damage to crops, the location of mineral deposits and earthquake faults, and more efficient means of surveying large tracts of land may save time, money, and even human lives. With superior definition of models for weather forecasting, climate and oceanic processes may eventually make possible more precise meteorological prediction and ultimately even weather control and global climate modification (Outlook for Space, 1976). Such an intelligent sensing system can play a dominant role in the activity of understanding the Earth as a dynamic physical entity, and can provide a major part of the ongoing monitoring of Earth useful in effective management of the individual and collective activities of man.

Second, NASA currently is obtaining and storing data from Earth-sensing satellites at a rate far out of proportion to the present or expected utilization of that data. The potential utility of collected data is not being realized because the raw data are not accessible in a timely and convenient manner, and because most potential users do not have the resources to extract useful information from the raw files. The current philosophy of data collection and storage had its origin in the early days of space research when sensors were sent into space, turned on, and all results transmitted back and stored. While this appears to maximize the utilization of the space vehicle, it has proven to be a false economy - the vast majority of uncategorized, generally unorganized data have never been and possibly never will be analyzed. The data format, its raw condition (digital conversions of analog sensor readings), and the complete lack of cross-referencing of contents make the data extremely difficult to find, interpret, or use. The tremendous volume of information already amassed and the expected increases in future rates of collection due to improved sensor technology make the philosophy of unorganized data acquisition obsolete. An alternate philosophy of goal-oriented data collection (information is gathered to meet specific objectives) was taken as the cornerstone of the proposed mission.

Thus, the main mission objective was to develop the concept of a flexible, intelligent, user-oriented automated information system for the collection, analysis, storage, and delivery of satellite Earth-sensing information (table 2.1). TABLE 2.1.- RATIONALE FOR DEVELOPMENT OF AN INTELLIGENT EARTH-SENSING INFORMATION SYSTEM. Why use remote sensing of the Earth Management ---- control Improved understanding - knowledge Information cannot be obtained any other way Current difficulties Vast amount of unorganized data Acquisition and distribution of useful information High cost The solution Goal-oriented observation Direct user interaction with the system World model-based observations Autonomous system Within rational cost bounds, the system should maximize the utilization of this information for the following purposes: scientific, managerial, commercial, and humanitarian. In addition, the collection and storage of data having little or no utility should be minimized, and the costs of acquiring, interpreting, and storing Earth resource information must also be reduced.

Inexpensive data delivery can be accomplished by a system operating with relatively little human intervention. Price reduction requires that images be processed without costly manual procedures, and that the physical satellite system be managed so as to obtain a maximum of useful data for the given configuration of orbits and sensors. It seems possible to design and construct, by the year 2000, a largely autonomous system that can directly interface with individual users in natural language, accept requests for information, and provide answers based on satellite observations coupled with a resident theoretical model of the state of the world. Such a system should be able to achieve sophisticated data interpretation at modest cost through advances in machine hardware and artificial intelligence techniques. Tabic 2.2 lists several desirable system characteristics and suggested methods for their achievement. The key to the proposed system is a sophisticated world model (section 2.3) that enables the system to perceive both the present state of the world and how that state changes in time. TABLE 2.2

DESIRABLE CHARACTERISTICS OF AN INTELLIGENT EARTH SENSING INFORMATION SYSTEM AND METHODS OF REALIZATION. Desirable system characteristics Methods of realization Cost minimization relative to level of service provided Maximize system autonomy Interface users directly to system Goal oriented observing relative to world model Wide utilization Flexible user interfacing including natural

language Automatic data interpretation AI techniques based on world model

2.1.1 Relationship to NEEDS

NASA has instituted the NASA End-to-End Data System (NEEDS) program, the goal of which is to improve the effectiveness and efficiency of the agency's data and information management methodology. NEEDS began with Phase I which addressed some very near-term data handling and processing problems. Phase II, initiated in 1978, concentrated on complete subsystems development to permit nearly real-time data management. Future Phase III will concentrate on low-cost communications and data distribution, and Phase IV will deal with integration of modular subsystems and systems techniques. NEEDS is a complex program which evolved on a problem-by-problem basis to accommodate everincreasing demands placed on it by the changing nature of the space program. A summary of Phases I and II projects appears in table 2.3.

TABLE 2.3 - NASA END-TO-END DATA SYSTEM (NEEDS) PROGRAM. Phase I 1. Synthetic aperture radar processor 2. Multispectral data processor 3. Digital data systems 4. Multipurpose user oriented software techniques 5. Resource effective data system definitions Phase II 1. Systems analysis and integration 2. Modular data transport systems 3. Information adaptive systems 4. Database management systems 5. Archival mass memory 6. Massively parallel processor

The intelligent Earth-sensing information system proposed in this report, and also the Titan mission described in chapter 3, will also place significantly increased demands on the present NEEDS program. This is not surprising as both may be viewed as parts of a natural evolution of present or near-term planned NASA missions. Moreover, with these new proposals as goals, the NEEDS program can implement Phases III and IV in a comprehensive fashion rather than on a problem-by-problem basis. The Earth-sensing mission demands are at the far extremes of present NEEDS activities, particularly in planning, scheduling, and control of satellite systems.

2.2 System Description

The system and mission goals described in this chapter are best summarized as an attempt to propose a flexible, ongoing tool of tremendous utility and sophistication. Most of the details presented are offered solely to illustrate one of many possible alternative approaches. The intent was not to prepare an encyclopedic discussion of design specifics but rather to indicate the general nature of probable solutions and provide sufficient subsystem details to permit preliminary technology assessment of the basic concept.

The Intelligent Earth-Sensing Information System (IESIS) has the following major features:

- An intelligent satellite system that gathers data in a goal-directed manner, based on specific requests for observation and on prior knowledge contained in a detailed self-correcting world model (section 2.3). The world model eliminates the processing and storage of redundant information.

- A user-oriented interface that permits natural language requests to be satisfied without human intervention from information retrieved from the system library or from observations made by a member satellite within the system.

- A medium-level onboard decisionmaking capability which optimizes sensor utilization without compromising users' requests.

- A library of stored information that provides a complete detailed set of all significant Earth features and resources, adjustable for seasonal and other identifiable variations. These features and their characteristics are accessible through a comprehensive cross- referencing scheme.

IESIS has five major components: (1) System/user interface, (2) uplink, (3) satellite sensing and processing, (4) downlink, and (5) on-ground processing. The basic- system is illustrated in figure 2.2. The user connects to the on-ground processor via a communication link and interactively defines his needs with the assistance of the system, accessing the database or directing IESIS to collect, process, and deliver information as required. The link might be a standard telephone line, and the entire transaction could occur in keyed natural language. Often, the user request should be answerable entirely from information already available in the system database, in which case IESIS appears to function much like any other interactive question-answering system (fig. 2.3).

Frequently, however, requests will require satellite observation data not yet available, in which case appropriate observations are scheduled by the on-ground processor. These instructions are uplinked via geosynchronous

 

Figure 2.2.—The basic intelligent Earth-sensing information system.

Figure 2.3.—IESIS response to user request for information already available in the system database.

 

communication satellites to orbital IESIS components that acquire and process the desired data and return them to the ground processor through a downlink (section 2.4.2). The ground processing unit further refines the information obtained, if necessary, then delivers it to the requester (fig. 2.4).

IESIS nominally operates with only one human in the loop - the user. This high degree of autonomy enables the system to be cost-effective and capable of rapid response. The sequence of operations during any user/IESIS interaction is outlined in figure 2.5.

IESIS has two basic modes of operation called background and foreground. The background mode of operation performs continuous goal-oriented observations of Earth and abstracts from these useful information for storage in a readily accessible, cross-referenced database. Background mode builds a broad scientific knowledge base that provides useful historical data at low cost for theory verification and testing. The foreground mode allows individual users to request that observations be taken and processed in nonstandard ways. The system must be sufficiently intelligent to help "naive users" obtain the information they want in an optimal or near-optimal fashion without restricting or unduly burdening the more sophisticated user.

2.2.1 Background Mode

In background mode IESIS continuously observes the Earth and gathers information to update the world model and to identify anomalies (sensor readings differing significantly from the expected). The system uses its world model to eliminate transmission of duplicate data and to implement basic principles of management by exception. The IESIS world model describes the topography and environment of Earth and can predict what a member satellite should record during its next observation period. During that period, the system collects data for "features" (e.g., lakes, forests, coastlines) and identifies all anomalies. Feature information is summarized to specify feature status without describing every pixel observed. For instance, if the height of a lake is known at its inlet and outlet, then the lake height at all points and flow rate can be determined. Only two pieces of data need be stored and transmitted by IESIS, rather than complete data for each pixel of the lake.

Anomalies are of two types. The first consists of transient normal events occurring at random, which are not to be permanently included in the world model. Examples are ships on an ocean, cars on a road, an iceberg, or a forest fire. IESIS should be capable of identifying such events by

SENSORY DATA ON BOARD PROCESSING NATURAL LANGUAGE INTERFACE

DATA BASE PROCESSING WORLD MODEL

Figure 2.4. - IESIS response to user request for information that requires satellite observation.

DATA ON DEMAND USER I LOGON I SYSTEM/USER INTERFACE - NATURAL LANGUAGE QUERY-RESPONSE ON GROUND PROCESSING - KNOWS ALL SATELLITES - KNOWS ALL USER REQUESTS - KNOWS ALL PRIORITIES - BUILDS PLAN FOR IMPLEMENTATION - GENERATES COST ESTIMATE PRIOR TO EXECUTION OF PLAN UPLINK I SATELLITE SENSING & PROCESSING - RAW DATA COLLECTION - MAKES DECISION ON DATA COLLECTION - ON BOARD PROCESSING - DOWNLINK - IMAGE DOWNLINK ON GROUND PROCESSING - PICTURE PROCESSING - UPDATE WORLD MODEL - ARCHIVE - DELIVER RESPONSE TO USER USER

Figure 2.5. - Typical data processing steps in any user/IESIS interaction.

their signatures. Observations of these anomalies may produce a sample count of the observation type, trigger an alarm, or generate a report of the incident automatically sent to persons who should be apprised of the situation. The second class of anomaly consists of unexpected, wholly novel events. Upon observation of such an anomaly (e.g., the eruption of Mount Saint Helens' volcano), all sensor data are returned to Earth for analysis, identification, and possibly, action. The expected anomaly file is updated to include the identity of the phenomenon together with directions on actions to take upon re-observation, if any.

Processed sensor readings for features encountered during an observation are archived. Archival data collected on the basis of features and their properties then may be used to improve world model accuracy or to build detailed models of particular features (e.g., Lake Erie) or types of features (e.g., fresh water lakes). Individually observed data points lose informational value over time and can be reduced to models such as a Fourier time series to retain more valuable long-term trends once sufficiently detailed surveys accumulate. While the importance of this aspect of data reduction will grow over time, the majority of data reduction is associated with the world model in the process of eliminating the storage of redundant data. The world model will enable individual features as well as groups of features to be studied and summarized easily.

2.2.2 Foreground Mode

The IESIS foreground mode allows individual users to make natural language requests for particular data to be collected and processed for their own purposes. The fulfillment of this request becomes a goal of the system. IESIS determines the appropriate sensor algorithms and requested data are acquired the next time the requisite sensors are within view of the feature or area to be observed. The system must ascertain that conditions specified by the requester are satisfied during observation (e.g., absence of cloud cover, proper sun angle). If they are not, IESIS informs the user and reschedules the run. Nonstandard data processing may be performed on sensor data with output in any user-specified format including terminal printout, photograph/hard copy, and so forth. IESIS must have default processing/output modes as well as a choice of several optimal preprogrammed methodologies. An unsupported user-written software library similar to that maintained by IBM also could be provided.

2.2.3 User/System Interface

The user/system interface illustrated in figure 2.6 has two basic functions. The first is to process data previously collected and stored on the database to provide desired output. The second is to schedule satellites to make new observations, process data obtained by system routines or user-written procedures, and to notify users when information is available for delivery in an appropriate format.

Individuals communicate with IESIS through a generally accessible information net. After valid identity is established, the user indicates via a natural language interface whether data retrieval or satellite scheduling is desired. If this choice is not already known, the system may be interrogated regarding relative costs in time or money of using the most recent file data or the next expected observations. Users requiring data retrieval may interact with IESIS to determine the type of information needed and to develop a carefully tailored retrieval and processing scheme. The user then ends the session or enters additional requests. Upon termination, system files and customer billing records are updated.

If a user desires satellite scheduling, the optimum method for obtaining required information including decisions about appropriate sets of sensors, observation frequencies, sites, and so on, is developed interactively. Customers are provided initial estimates as to the probable time of completion and expected costs for data collection, and may revise or cancel a request on the basis of these preliminary appraisals. Each such interaction updates the request schedule so that observing satellites can perform appropriate observations and deliver derived data to the correct user after processing. Individuals may access the system to cancel or modify a previous request at any time up to the actual taking of data. Further, IESIS can accept requests for time- or event-oriented measurements such as photography of a particular area every month, or for observations contingent on specific events or conditions recognizable by the system such as volcanic eruptions or forest fires.

DETERMINE FEASIBILITY; OPTIMUM SENSORS, PROCESS, COST AND TIME ESTIMATE

? MODIFY V REQUEST / DISPLAY \ ( TO

(USER CONNECTS TO SYSTEM VIA ^NETWORK

IESIS REQUEST SCHEDULE AND/ PROCESSING REQUIREMENTS\ FILE UPDATE REQUEST/ SCHEDULE AND/ PROCESSING I REQUIREMENTS^ FILE N.

Figure 2.6. User/system interface.

2.2.4 Natural Language Interface

The primary reason for a natural language interface is to allow the largest possible number of novice users to access IESIS directly without need for "interpreters." The principal advantage for more knowledgeable customers is convenience, though many customers will likely require a more formal and precise interface language in addition to the natural language capability. There are major problems with current natural language interfaces that require careful consideration. The two primary difficulties are:

(1) The machine has only a very rudimentary notion of the subject of conversation during sentence interpretation even if it is quite competent to answer questions posed in a formal query language.

(2) The machine is not a social being and thus is deaf to most of the subtle information content in sentences generated by people.

Such flaws exist in all current natural language systems. For the anticipated "naive" user these obstacles must be minimized or IESIS will prove extremely uncomfortable and time-consuming. These flaws could possibly lead to system avoidance by less-sophisticated users, thus defeating one of the major mission goals.

Perhaps the simplest way to overcome such problems in natural language systems is to restrict the domain of discourse to a small set of possible concepts keyed to known individual human differences. Ultimately, the following may be the best approach for the "naive" user: Each would have a personal identification code known to IESIS, permitting the system to adjust its language to a compatible dialect. Knowledge of customer category could enable IESIS to employ reasonable default assumptions in restricting the domain of discourse (and thus the vocabulary) to a manageable subset of the overall system domain.

It is important that IESIS be able to communicate at the appropriate level of complexity and brevity. To accomplish this requires a system capability of modeling individual users. Some initial work in this area has been done (Rich, 1979), but no known current technique offers the level of performance necessary for IESIS. Natural language interfacing is one area that requires considerable advancement before it can hope to meet the IESIS system requirements: domain model, user model, dialogue model, reasonable default assumptions and common world knowledge, and explanatory capabilities.

2.2.5 Artificial Intelligence Problem-Solving

Clearly IESIS presents the usual difficulties in problem- solving typically involved in AI question-answering systems. But there is another new and important dimension added - effective combination of a world model database, world model theory, and a potentially resource-limited observational capability. The power of the problem-solvers and planners, and their capacity to execute plans in a dynamic and only partially known environment, will be instrumental in achieving a high-quality information delivery service at minimum cost.

Two specific areas where the quality of problem solving affects overall system efficiency and cost were considered for illustrative purposes. The first is communication link capacity. Given the goal of answering a large number of information requests, an intelligent planner able to isolate a parsimonious set of observations can considerably reduce ground link and intrasatellite link volumes. This set of observations is determined by consideration of individual requests, the ensemble of all current requests, and a prediction of expected requests.

A second area of concern is the number of satellites. If the system can employ a very sophisticated theory of observation, then it may be possible to shift most data-taking tasks to lower resolution. This system would allow data- taking by orbiters at higher altitudes having greater fields of view; thus a smaller total number of satellites would achieve the same frequency of coverage.

A major IESIS goal is to perform appropriate automatic data interpretation. System success in this activity demands a high-level capability to understand relationships between sensor readings and the actual state of the world as defined by human-oriented descriptors. This is precisely the problem in visual perception, an active current area in the field of artificial intelligence. Section 2.2.6 further discusses several aspects of the perception problem for an Earth- sensing system, and section 2.2.7 describes the need for flexibility and adaptability in IESIS. In both areas - perception, and system flexibility and adaptability - there is a tremendous need for development of fully autonomous techniques far more powerful than those presently available.

2.2.6 Theory of Observation

While the number of distinguishable states of the world of human interest (at a particular level of resolution and description) is extremely large, this figure is still dwarfed by the vast number of distinguishable ways the world may appear to rudimentary sensors. Machine sensors and human eyes see entirely different things when minor changes in the world state occur. For instance, in hilly terrain at low sun angle, satellites sensor readings vary rapidly as the shadows progress, but most of what is of interest to human beings is invariant.

To extract interesting information from sensors it is necessary first to understand the nature of the sensor as a transducer so that a mathematical inversion process can be performed on the readings. This involves computation of the electromagnetic reality at the image sensor location (i.e., at the satellite, but before the sensor transduces visual photons into a signal). This extracted information is called "satellite local reality." If satellite local reality is not obtained, then an interpretation correlation problem arises when different sensors are used at various times to observe the same thing.

Once satellite local reality has been determined, it must be mapped onto an "object local reality" where "object" refers to the surface or volume under observation. For example: suppose that the state of the relatively clear atmosphere and a power spectrum at several chosen frequencies for a particular image pixel element of the Earth's surface are accurately known. The theory of observation must be able to use the world model (in this case the atmospheric component of the model) to determine which parts of the satellite local power spectrum are generated from a ground effect, and which are atmospheric phenomena. Reflection, refraction, absorption and emission cannot, in general, be clearly separated. Thus, the attempt to translate satellite local to object local descriptions requires some additional information.

The predictive aspects of the world model are important here. The model can predict much of the satellite local power spectrum, so inversion to object local spectrum need not be performed at all except in specific (and hopefully, relatively infrequent) cases where observations deviate significantly from prediction. These deviations are called anomalies. When an anomaly occurs, there may be simple alternative world states IESIS can hypothesize in an attempt to find an explanation for the anomaly. In this mode of action the model is altered and a new prediction for the observation generated. If a reasonable world model alteration leads to a predicted image that matches that actually observed, then the altered model may represent an adequate estimate of current reality. Other specific observations should be designed with the objective of testing the new hypothesis.

If an anomaly cannot be disposed of by hypothesizing new world states, alternative mechanisms are needed. It is desirable to proceed as far as possible without explicit human intervention. One alternative is to automatically schedule other observational configurations (different satellites, sensors, or lighting conditions) to gather enough satellite local information and permit clear computation of object local signature. Once this is accomplished the final interpretation must be made which involves mapping object local signatures into a state description suitable for incorporation into the world state component of the world model (see section 2.3).

The role of a predictive model in efficient image gathering is essential. Even when it is theoretically possible to clearly map satellite local reality into a high-level description of object local reality without such a model, model use can significantly increase the efficiency of the observation and the speed of the interpretation process. The following is a partial list of the many ways world models may be helpful:

? Extending the range of possible viewing conditions under which usable information can be gathered (e.g., compensating for variations in sun angle) ? Predicting when certain types of observations are impossible because of unfavorable viewing conditions which can be known prior to the time of observation ? Computing the least costly set of sensors (e.g., fewest sensors for shortest time, or use of sensors which at the particular time have no other demands on them) needed to determine a particular fact about the world ? Avoiding taking certain new observations by deriving at least some responses to requests from information already in the world model database.

IESIS is not oriented toward the storage of information as images. Rather, images are processed in real time (or almost real time) and only extracted information is stored in the world model. In such a context, the emphasis shifts from finding observation strategies which yield absolutely unique sensor signatures for identifying the condition of the world, to observing only what is necessary to identify the state of the world in the context of the world model. The theory of observation can be viewed as part of the world model, and represents a large part of the knowledge necessary to connect sensor-encoded information with more human-oriented descriptions of reality contained in the world model database.

2.2.7 System Flexibility

It is very difficult to anticipate the entire range of users to whom Earth-sensing systems may be applicable. IESIS must be flexible enough to allow a scale-up of total system throughput to accommodate a growing number of customers. Similarly, it is unlikely that the mission system will have available, by the year 2000, the ultimate in sensor technology. Almost certainly, a rapid evolution of ideas and technology will occur after a short period of system use. If the system is not to be the seed of its own rapid obsolescence, it is imperative that it be flexible enough to accommodate new modes of observation including new equipment and new processing procedures.

The general philosophy of providing a flexible information system for the sophisticated user virtually demands that the user be able to specify new algorithms for controlling the data accumulation and data-analysis processes. User-defined data collection control becomes important for the advanced user when observation scheduling must be sensitive to dynamic events in real time - where it would be impractical or impossible to use the standard system- scheduling mechanisms. Such individuals may require specialized data interpretation processing for a variety of reasons including the needs for recognition categories not defined as standard categories or for specialized data displays.

The argument can be made that, besides the general desirability of system flexibility, it is important to give advanced users a flexible, complex, and adaptive tool. Very likely some of the most innovative and important IESIS applications will arise through the efforts of such individuals.

At present, Landsat data customers represent a relatively small but sophisticated population comprised mainly of engineers and scientists. IESIS is intended to reach a much broader spectrum of potential users, the majority of whom are "naive" with regard to computer technology. It is imperative that a reasonable model of this target population be generated as the system is implemented. Such knowledge is necessary for detailed design of user/system interfaces, ensuring insertion of user-related elements into the world model, and for determining signature analysis and pattern recognition techniques required to answer probable user questions.

2.2.8 Data Archiving and Compaction

The traditional NASA information gathering philosophy has been to collect as much raw data as possible from each mission and then allow university, industrial, and government researchers to complete the analysis. In the early days of space exploration this strategy was reasonable, based on spacecraft investment, insofar as it maximized return. But today, advancing satellite technology has greatly expanded the number of sensors flown and available data rates. The resulting torrential flow of information has overwhelmed the capacity of the system - only 0.05% of data collected from space has ever been analyzed. The great unused bulk of observations must be stored even though much of it is of marginal quality (e.g., obscured by clouds) and probably never will be analyzed.

NASA should consider revising its philosophy of data collection to: (1) Make use of knowledge gained from previous sensing missions to reduce redundancy, (2) adopt a goal-oriented approach to Earth-sensing and other observational missions, (3) begin to identify and dispose of poor- quality data, (4) condense information as it ages and becomes less useful, and (5) present data with full indexing and cross-referencing to maximize their utility to the consumer.

Knowledge and experience combined with artificial intelligence techniques can eliminate redundancy. For example, it is extremely inefficient for an Earth-sensing satellite to "rediscover" a lake, road, or city on every orbital pass. The truly important aspects of the object are its fundamental attributes area, temperature, color, texture, etc. - many of which are either constant or predictable. The use of a world model to eliminate continual rediscovery of such features could greatly reduce the extraordinary redundancy of most visual imagery.

All object attributes studied must reflect worthwhile goals. One goal should be the assembly of a historical record of Earth features. Others may include more specific user-defined objectives. This new emphasis on goal-directed observations does not preclude data utilization by the scientific community in the investigation and verification of new theories; quite the contrary, it should actually enhance this activity by enabling researchers quickly and easily to direct IESIS to collect and process data under closely controlled conditions.

Many time-oriented observations lose some of their value with age. After an extended period of time, long-term trends are much more useful than individual data values. In the proposed IESIS system only the long-term trends are retained original data are eventually discarded. Thus, the system processes all data immediately for specific goals and, at a later time, integrates trend information into a more compact world model representation such as a long-term temperature gradient.

As data are collected in orbit, Earth features and their processed image characteristics must be fully indexed to sort features and characteristics and to analyze them by group. The data then may be manipulated from within a fully cross-referenced base. For instance, area type can be called out and summed to obtain the total rye acreage in a given state. This cross-referencing feature is critical to the effectiveness of the Earth-sensing information system. The ability to automatically cross-reference and access data by content and feature allows rapid aggregation and correlation, and may promote new research as rapid access to useful scientific data becomes routine. The proposed database is organized using geographic location as the primary key (similar to the World Reference System used for Landsat data) with individual features also keyed. Feature keys greatly simplify the generation of inverted files listing, say, all lakes, deserts, wetlands, forests, or housetop areas obvious widespread applications. The detailed mechanisms of records layouts, file structures, and database languages are beyond the scope of the present study and are recommended for future investigation.

The expense of storing data is a very significant part of computer system cost. This consists of direct charges for storage media as well as the costs of transferring data to and from local and remote storage devices. Data compaction and compression produce cost savings by reducing storage and transmission requirements. In addition, these data reduction methods enable more efficient information retrieval and more economical transmission of large quantities of data over computer networks.

There are no standard definitions for compaction and compression, so the following usages are adopted for the purposes of the present study:

Compaction of data - any technique that reduces the size of physical data representation while preserving the relevant information.

Compression of data the application of some function to elements of the database. If x is a specified element of the database then the compression of x is v, where v = /fx). Usually /is invertible, which means that the original information may be recovered whole from the compressed data.

Compaction techniques other than compression involve elimination of information deemed superfluous, in order to decrease overall storage requirements. One such method is abstraction. Abstraction is accomplished by processing data over important common image features (in the case of photographic information) by using, for instance, a world model. After abstraction it is not possible to recover the original image.

Mission data such as are received daily or are already stored in various NASA facilities (e.g., the EROS data center at Sioux Falls, South Dakota) and slated for compaction may be classified within two broad categories - continuous and noncontinuous depending on timing and event characteristics. Classes of continuous data include:

? Periodic data When the same event appears again and again, only one copy need be saved.

? Trendless data If the data are random continuous, a sample should be taken to check for trend. If no trend is found, the data may be represented by a histogram updated regularly as more information accumulates. Only histogram parameters need be saved.

? Data with trend If a trend is detected, multiple regression and curve-fitting are best to record the feature. It may be possible to correlate variation in one time series with that in another (e.g., how sea level is affected by temperature or pressure). Data compression is achieved in this case by fitting polynomial segments, possibly straight lines, to the data.

? Data with turning points Turning or "inflection" points, such as where an upward trend suddenly shifts downward in a time series graph, may require different models to be fitted to the two parts of the series. Only the parameters of fitted polynomials and a few statistical abstractions (e.g., maxima and minima, mean, variance, and several others) in any particular range need be saved.

Suggested classification and processing techniques for non- continuous numerical data are summarized in figure 2.7.

2.3 The World Model

The world model is a crucial element in the achievement of specific goals. Without a sophisticated model two serious problems are encountered with remote Earth-sensing data, particularly images:

? It is very difficult, if not impossible in many instances, to accurately separate interesting from non- interesting observations.

? It is difficult to comprehend raw sensor data in terms readily understandable by human beings.

The first of these leads to the collection and retention of great volumes of data, simply because there is no practical way to perform an appropriate selection of the useful subset of information applicable to a user request. The second problem results in gross underutilization even of potentially useful data. The lack of a world model in present-day spacecraft makes necessary a voluminous and costly stream of highly redundant data which must be transmitted and collected on the ground before any useful information is retrieved, leaving a huge reservoir of unprocessed data in expensive storage facilities. It is the world model which transforms IESIS from a collection of remote cameras into an entity able to perceive the planet in a manner interesting and informative to humans. This world model is a compact representation of persistent spatial and temporal characteristics of the Earth (its land, oceans, and atmosphere), and algorithms for use of the model.

2.3.1 World Model Structure

The IESIS world model has two separate components. The first is the state component, which defines the physical status of the world to a predetermined level of accuracy and completeness at a specified time. Second is the theory component that allows derivation of the following information from the state component: (1) Values of parameters of the world state not explicitly stored in the state component, and (2) a forecast of the time evolution of the state of the world. The theory component gives the world model a predictive capability, in that the model can predict facts about the world not explicitly retained in the database.

The disparity between predicted information and reality generally increases with increasing computational distance separating the starting information and the derived result, increasing time in the case of forecasting, and certain other factors. The world model requires a continual influx of new observations to remain temporally current. A major research goal for efficient IESIS operation is to develop the AI capability to construct an effective real-time world model which can act as a database for answering questions

Figure 2.7.—Classification of and suggested processing techniques for noncontinuous numerical mission data.

about the state of the world and as a predictive mechanism for controlling observation satellites and interpreting observations.

The world model database must contain state component information about the expected character of points on the Earth. This includes land use (crop type, urban type, etc.) and ground topography. The world model theory component must predict some rather ubiquitous changes that occur, such as alterations in foliage color and foliage loss for certain vegetation areas as a function of season and precipitation history; ice formation and melting with the seasons; and variations in appearance of rivers from flow rate changes due to watershed runoff. As with the ensemble of observing satellites, it should be possible initially to set the Earth-sensing system into operation with a limited world-modeling capability, and later expand this model as the technology progresses. 103 pixels

AVERAGE 30 niches 15 CHARACTERISTICS 1 SENSOR VICTORIA, AUSTRALIA E141-45 10J 106 pixels VS. 450 CHARACTERISTICS 2,222 : 1 REDUCTION

Figure 2.8. - Sample niche features map of Mildura Gty, Victoria, Australia.

The key element in handling, processing, and storing data in the proposed satellite system is the use of a world model to abstract useful information from images and thus accomplish a large reduction in required data transmission and storage. The model gives the satellite "experience" by which to judge new observations. Data compression has been investigated in video imagery and, in some cases, compressions of 20:1 have been accomplished, but not at acceptable fidelity (Graham, 1967). It has been found that methods using feature extraction without benefit of a world model are capable of compression ratios in excess of 100:1 (Chien and Peterson, 1977). A world model permits still greater data compression by interpreting features and their established properties in the full context of the known land, sea, and atmospheric environment of the Earth.

The simplest world model is a flat land map. Figure 2.8 illustrates such a map of Mildura City in Victoria, Australia, representing an area encompassing 16 features in 5 distinct land types: (1) river, (2) lake, (3) forest, (4) cropland, and (5) city area. Each feature is termed a "niche" ? a surface feature on land or ocean possessing a relatively clear boundary, common features within that boundary, and whose location does not change rapidly with time. On land, niches are closely spaced, whereas in the oceans and atmosphere they may be quite large. Table 2.4 gives general characteristics of land, ocean, and atmospheric world models.

TABLE 2.4,-WORLD MODEL CHARACTERISTICS OF LAND, OCEAN, AND ATMOSPHERIC WORLD MODELS.

Land Extensive, detailed contour maps exist Relatively static Sharp boundaries Small feature size Widely populated Oceans Gross maps exist Constant altitude Wide boundaries Large feature size Sparsely populated Atmosphere 3-dimensional Elementary model exists Rapidly varying Boundaries difficult to identify Large shifting patterns Some small feature sizes Substantially unpopulated

Niches possess a high degree of spatial redundancy. A large lake remains the same lake on each satellite overpass, and parts of the lake are very similar to other parts, yet are very distinct from the surrounding land. The large redundancy permits very substantial data reduction by processing each sensor reading across a single niche and extracting desired characteristics of that niche. These characteristics can be combined across sensors to produce yet further compaction. Despite the large reduction in physical data, no useful information is lost.

The likely scale of data compaction is illustrated by considering a 10 km X 10 km land region at a pixel resolution of 10 m. This scale gives an image measuring 1000 X 1000 pixels. If just one observational wavelength is involved, and 8 bits are used to represent intensity at each of the 106 pixels, the resulting image is 8 X 106 bits. A limited examination of aerial photographs suggests such a region will possess an average of 30 niches. Each must be fully described in terms of characteristics important to it such as area, average sensor value, variance, higher moments, two- dimensional sensor intensity gradient, and texture. If 15 characteristics are sufficient to describe most niches, then only 3600 bits (15 characteristics X 30 niches X 8 bits/niche-characteristic) are needed to replace the 8,000,000 bits of the full image. A reduction of 2222:1 is immediately accomplished.

Further reduction can be achieved by combining data across the approximately 20 sensor wavelengths proposed for the satellites, and also across the 15 characteristics. This reduced data can be used, for example, in signature identification, specification of niche status, or for classifying an anomaly. Near-maximum reduction occurs when imagery is required to answer a sophisticated question posing a choice from 256 (= 28) alternatives, an answer requiring just 8 bits - the 106 pixel elements over 20 sensors demand one 8-bit transmission and subsequent storage for a reduction of 20,000,000:1 ([106 elements X 20 sensors X 8 bits/sensor- element]/[8 answer bits]). If the question requires a "yes" or "no" answer only 1 bit must be transmitted and the absolute maximum data reduction in this simplified example is achieved - 160,000,000:1 - as summarized in table 2.5.

IESIS also is capable of discovery. Novel occurrences can be detected by the satellite system when searching for anomalous features in the imagery as compared to the world model. Many of these anomalies may be boundary changes in the existing map, e.g., overflow of the River Murray as shown with dark lines in figure 2.9. Others will involve unusual values of sensory characteristics of the niche (e.g., blight on a corn field). Prompt identification of these anomalies allows real-time management action in response to the "abnormal" occurrence. Figure 2.10 illustrates a hypothetical set of readouts from an intelligent satellite scan over Mildura.

The most efficient use of a world model requires placement of a simplified model in memory onboard the satellite system (to accomplish direct data reduction) and retention of a more sophisticated model in the ground operations facility. The latter serves as a master Earth model for use in updating, calibrating, and further processing transmitted data. (Estimated memory requirements are given in section 2.3.2.) Another very important feature of the ground- based world model is that it allows full cross-indexing of

TABLE 2.5,- DATA REDUCTION USING WORLD MODEL IN A SIMPLIFIED EXAMPLE.

Transmission/storage task Total bits Net reduction 106 image pixels, 20 sensors 8 bits each 1.6X108 1:1 30 niches, 20 sensors, 15 characteristics at 8 bits 7.2X104 2,222:1 256 choice answer 8 20,000,000:1 Yes or no answer required 1 160,000,000:1

103 pixel?

AVERAGE 30 niches CHARACTERISTICS 1 SENSOR VICTORIA, AUSTRALIA 106 pixels VS. 450 CHARACTERISTICS 2,222 : 1 REDUCTION Figure 2.9. - Sample niche features map of Mildura City; alteration in boundaries of the River Murray due to flooding.

terrestrial niches, sensor characteristics, and subsidiary characteristics. The ground model thus constitutes a full working library. A land agent, for instance, easily could retrieve moisture content on all corn acreage in southwest Iowa from the master files.

In addition to a physical database memory onboard and on the ground, the world model requires extensive artificial intelligence software including expert systems which use the database in controlling sensing, image location and rectification, data processing, labeling, anomaly search, and decisionmaking on board the satellite and on the ground. A set of expert systems is needed to handle overall coordination, observation scheduling and user-generated processing tasks (fig. 2.11).

2.3.2 Onboard Memory

Present predictions for onboard memory in the year 2000 for Earth-sensing satellites are on the order of 1014 bits (Opportunity for Space Exploration to Year 2000. Address delivered by A. Adelman at Goddard Space Flight Center, 1980). In this section, these projections are compared with the storage capacity required for IESIS abstracting and onboard processing. The following estimates emphasize image processing because this is the type of data transmitted at the highest rates - on the order of 650 Mb/sec for SAR and 320 Mb/sec for the multilinear array (Nagler and Sherry, 1978).

The bits stored aboard the satellite or satellites for use in immediate processing tasks presumably are substantially less than the number stored on the ground in IESIS. The on-ground world model is a continually updated version of the Earth model. Future feasibility experiments plus new developments in computer science and technology will dictate the specific allocations of memory required in space and on the ground. Hence, the following are only crude estimates of the storage requirements under a range of plausible assumptions.

For the present work it is assumed that a correlation is to be performed between the incoming image and its image description stored in the onboard world model. Significant

PRINT OUT IMAGE COORDINATES: E141-45 S30-06 LOCAL NAME: SEC 192, MILDURA, VICTORIA, AUSTRALIA 9-12 - 74 21:08 VMT NICHES 1 2 3 4 LAKE RIVER FOREST GLEN GRASSLAND CLASS NAME GEORGE CHARLES STIRLING GREEN LOW CHANNEL # AVERAGE 1 2 3 4 5 6 SENSOR BLUE GREEN YELLOW RED NEAR/R FAR/R READING: 85 ? 7 35 ? 3 7 ? 1 1 ? 1 6 ? 1 8 ? 1

63 ?4 60 + 2 30 ?3 16 ? 5 5 ? 1 1.6 ? 5

7 ? 1 58 ? 7 25 ? 2 12 ? 2 8 ? 2 3? 1 'COLOR 5960 ? 38 4500 ? 26 3106 ? 22 H2 ' IR 310 ? 28 305 ? 6 316 ? 5 H3

WEIGHTED AVERAGE OVER CHANNEL: COLOR TEMP 4302 ? 20 GROUND TEMP 307 ? 02 MOISTURE 0.06

H4

WOMBAT COUNT 506/mi2 CLOUD COVER 15% MURRAY IN FLOOD

H5

HIERARCHY HI ALL DATA, ALL SENSORS H2 NICHE AVERAGE, ALL SENSORS H3 NICHE CHARACTERISTIC H4 NEIGHBORHOOD CHARACTERISTIC H5 SPECIAL REQUEST

Figure 2.10.~ Hypothetical IESIS user printout following scan of Mildura City.

features of the incoming image are contrasted and identified with distinguishable features (niches) catalogued in the model. The number of bits N needed to store this information on some or all features is N = ni, where n is the number of niches in all or part of the world model and i is the number of bits required to describe a niche. Examples that estimate N are given below.

Niches. The niche is a broad distinguishable geographical region having some common features across its surface when viewed from space. Niches are easily separable, somewhat permanent features (large rivers, canals, lakes, major highways, cultivated areas or forests) recognizable within a predetermined orbital swath. Since niches consist of common features, data acquired across them have very high redundancy. A large amount of data reduction is obtained by describing the entire niche with a limited number of common values abstracted from the whole set of niche data on record.

It is estimated that a total of i= 122 bits is required for individual niche identification, as follows: Location of centroid, 6 bytes, 48 bits; maximum and minimum of horizontal, 2 bytes, 16 bits; maximum and minimum vertical, 2 bytes, 16 bits; orientation, 2 bytes, 16 bits; abstract shape among 1000 choices, 10 bits: naming the niche, 2 bytes, 16 bits. In addition, s sensors are assumed scanning

GROUND

? CHECK LOCATION OF SATELLITES ? PREPARE NEW WORLD MODEL DATA, OVERALL MEASUREMENT ROUTINE FROM EXPECTED CLOUD COVER, WEATHER, SUN POSITION, ETC., FOR EACH SATELLITE IN UPCOMING MEASURMENT ? TRANSMIT DATA AND INSTRUCTIONS ? RECEIVE SATELLITE DATA ? PROCESS UNIDENTIFIED ANOMALIES AND TAKE APPROPRIATE ACTION ? UPDATE WORLD MODEL, HYPOTHESIZE SEASONAL AND TIME DEPENDENT PATTERNS, DETERMINE NEW CORRELATIONS AND TRENDS ? INTERFACE WITH USER AND DATA ARCHIVAL SYSTEMS AND NATURAL LANGUAGE INTERFACE ? COORDINATE AND MAINTAIN ALL SYSTEM COMPONENTS ? DETERMINE SYSTEM STATUS AND PERFORMANCE

SATELLITES ? DETERMINE LOCATION FROM AUTONOMOUS NAVIGATION SYSTEM USING POSITION SENSITIVE ANGULAR MEASUREMENTS TO CELESTIAL OBJECTS, NICHE LOCATIONS AND MEASUREMENTS TO KNOWN BEACONS AND THE GPS ? CALL OUT NICHES BY STRIP ALONG OVER PASS ? APPLY TEMPORAL, SENSORAL, SUN AND OTHER CORRECTIONS ? FROM NICHE TYPES AND BOUNDARIES DETERMINE OPTIMUM SENSOR COMBINATIONS, ADJUSTS MEASUREMENT STRATEGY AND ALGORITHMS ? APPLY OTHER PREPROCESSING ? PROCESS DATA ? CHECK RESULTS ? USE RESULTS TO DETERMINE NEW LOCATION ? APPLY POST PROCESSING PROCEDURES ? DETERMINE TRANSMISSION ROUTING TO GROUND

? PREPARE FOR SPECIFIC SATELLITE ORBIT ? CHANGE ALGORITHM ? CHANGE MEASUREMENT STRATEGY ? CHECK RESULT ? CHANGE MODEL ? CHANGE DATABASE

Figure 2.11.- Functions of IESIS expert systems.

various bands covering a niche having ? characteristics coded in memory. Thus, 8 sk bits are needed for niche sensing, and 20 sensors having 15 characteristics require 2400 bits. It is also conceivable that 10 other characteristics might be detailed yielding an additional 80 bits. Consequently, without using boundary analysis in the image correlations, the total information i required for each niche is approximately 2600 bits, more than an order of magnitude larger than the information needed simply to identify the niche.

Thus, it is most efficient if the onboard world model simply consists of descriptions of those niches the IESIS satellite(s) will encounter along an orbital swath on the next revolution. The model is uploaded from the ground prior to each pass. If swath width in kilometers is W, repeat time (in days) for the satellite to cover the entire equatorial

Figure 2.12,- Ground-based world model data processing operations.

region is T, and the number of passes per day (generally 15-16) is/, then, taking the Earth's circumference roughly as 4X10" km, T = (4X104)//W. If n denotes the number of niches in one pass for a given swath width, then n = (4X104)W/niche area.

For example, if swath width is 330 km and the satellite executes 15 rev/day, then T is on the order of 8 days. If the area of a representative niche is roughly (for simplicity) 3.3 km2, then n = 4X106 niches and N = ni = (4X106) (2600)= 1.04X1010 bits.

Niche at each pixel. A gross theoretical estimation is obtained by assuming that each pixel element is associated with 8 bits of information of t different types. Thus, each pixel element plays the role of a niche. If A is the resolution of each pixel in km2 and E is the approximate surface area of Earth (5.15X10? km2), then the total number of bits required to describe pixel elements covering the entire planet is %tE/A. If pixel resolution is 10 m, then A - 10"4 km2/pixel which gives

(5.15X108 km2)/(10~4 km2/pixel) = 5.15X1012 pixels

for all of Earth. The total information required to store this gross world model is of the order

8tE/A = 8/(5.15X 10?)/10~4 = 4.12X 101 3 / bits.

It appears that a memory capacity onboard a satellite of the order 10'1 bits is sufficient to allow system operation on an orbital swath basis. Presumably between 1014-1015 bits are needed for ground memory. It is estimated that by the year 2000 roughly 10'4 bits of in-space memory will be available, so these requirements do not appear particularly stringent (Opportunity for Space Exploration to Year 2000. Address delivered by A. Adelman at Goddard Space Flight Center, 1980).

2.3.3 Ground-Based World Model Processing

Processing at the central ground facility can be broken into paths as shown in figure 2.12. Action items which the system should generate in response to observation of an anomaly are carried out, appropriate alarms sounded and individuals notified. Anomalies that the system cannot USER REQUESTED DATA ABSTRACTED DATA

identify undergo close human scrutiny - if subsequently identified, the expected anomaly/action file is updated; if not, the event is catalogued and correlated with other similar but also unidentified anomalies. Identified anomalies keyed to time of occurrence, intensity, and other important parameters are archived and used to update the world model if necessary. Data abstracted from observed features are archived and used for world model updating. Information gathered in response to user requests is transmitted to customers for their exclusive use and is not automatically stored by the system.

2.4 Autonomous Satellites

This section focuses on individual satellites and the collective IESIS system, as suggested in figure 2.13. To satisfy overall IESIS goals, all satellites must be equipped with an appropriate ensemble of sensors and reside in orbits providing sufficiently frequent observation opportunities for all points of interest. Just as important, however, is the ability of each device to accept brief high-level instructions to guide its observations, and to perform massive onboard processing for abstraction of high-level information.

2.4.1 Onboard Processing

The observing satellite receives the schedule for its next pass from the uplink (section 2.4.2) and performs the requisite processing according to this instruction set. Terrestrial cloud cover and weather conditions are obtained from one of the IESIS geostationary satellites and navigational data are transmitted from a global positioning system already in place. Each satellite adjusts its attitude as required, turns detectors on and off, modifies sensor resolution, and takes both active and passive observations of the Earth. Data then are processed by comparison to predicted observables as derived from the world model.

Anomalies are identified if possible, catalogued, and appropriate action taken. Anomalies that cannot reliably be identified by IESIS are placed on file for transmission to a human analyst for interpretation and action. All sensor data are abstracted and summarized by feature and placed in the abstracted observation file for archiving and world model updating. Any user-requested processing is then performed and a user file established. These processes are shown schematically in figure 2.14.

2.4.2 Uplink and Downlink

The uplink sends detailed observational and processing schedules to IESIS satellites. A possible component of the uplink package is a set of sensor values expected to be observed during the next set of observations. This expectation is generated by using the world model as the standard against which to compare sensor observations so that anomalies may be identified. The uplink is illustrated in figure 2.15. Information needed during the next observation period is transmitted from the central processing facility via ground station and geosynchronous communications satellite up to the observation satellite as discussed in section 2.2.

NAVSTAR - GPS GEO-SATELLITE TDRSS LEO - EARTH OBSERVATION SATELLITE

IESIS

Figure 2.13,- IESIS satellite system.

LEO - EARTH OBSERVATION SATELLITE

The downlink reverses the uplink process by consolidating data gathered by satellite and returning them to the ground in one transmission. The downlink also uses the geosynchronous communications link to transmit data to a ground station which relays the information to the central processing facility. The process of downlink consolidation is illustrated in figure 2.16.

2.4.3 Image Processing

A primary IESIS requirement is the necessity to perform rapid and massive data reduction aboard the satellite in the sequence suggested in table 2.6. The focus is on image data acquired at high rates ? presently 120 Mb/sec for SIR, 85 Mb/sec for the Thematic Mapper, and in excess of 600 Mb/sec for SAR systems (Nagler and Sherry, 1978). Such rates may arise in each of perhaps 20 sensors in certain extreme cases of IESIS operations, which requires that the spacecraft carry onboard high-speed processors.

IESIS high-speed processors might evolve from faster serial logic devices, e.g., those which may be developed from Josephson tunnel technology. However, an alternative approach is the use of parallel logic to perform many operations simultaneously. By executing thousands of computations at once an intrinsic speed advantage equal to the number of individual processors is theoretically possible. In practice, this hypothetical limit may not be attainable due to pragmatic technological restrictions on each individual element in an array of thousands of processors. In spite of this, computing speeds within two orders of magnitude of the theoretical limit have already been obtained (Schaefer, 1980). Data handling using thousands of active elements simultaneously is called "parallel processing." Preprocessing functions including extremely rapid rectification of images, formatting, noise removal, imbalance, and radiometric correcting can be performed best by manipulating data in parallel rather than serially. Projections for serial onboard image processing range up to 109 operations per second (Opportunity for Space Exploration to Year 2000. Address delivered by A. Adelman at Goddard Space Flight Center, 1980), whereas, those for parallel processors extend up to 1011 operations per second per pixel (assuming 100 clock cycles per operation). Another capability expected to be using parallel processing is a 1000 X 1000 parallel input array operating at up to 100 MHz clock rates (James Strong, personal communication, 1980). In one proposed 1000 X 1000 input array, as shown in figure 2.17, threshold photosensors on a wafer lead directly to massively parallel processing elements which in turn are connected (also through the wafers) to memories. Processing rates expected for massively parallel

SENSOR DATA UV VIS NEAR IR FAR IR SAR LIDAR ADJUST SENSORS

ACQUIRE DATA

CONVERT A TO D

PRELIMINARY

ANALYSIS FOR OBSERV. QUALITY REQUEST CAN BE MET ABORT NEXT PASS SCHEDULE AND WORLD { MODEL SEGMENT J4LE REQUEST CANNOT? BE MET

REGISTER MODIFY OR

RECTIFY ABORT

PROCESS

PROCESSING

COMPARE TO WORLD MODEL

Figure 2.14. - Onboard processing flowchart.

processors (MPP) are tabulated in table 2.7. Further information and technical descriptions are available in Gilmore et al. (1979).

TABLE 2.6. ONBOARD DATA REDUCTION PROCESSES.

Preprocessing Name image region Format image data Apply sensor corrections

Processing Cross-correlate image and map Geometric correction Resample data Obtain niche boundary Generate niche mask Process niche data Combine data across sensors Detect and characterize anomalies

Postprocessing Attach niche labels User tags Assign priorities Update data base

TABLE 2.7. ESTIMATED SPEED OF COMPUTATIONS USING MASSIVELY PARALLEL PROCESSOR. Function

Function 108 Hz clock Area sec Average 50 sec Variance 0.11 msec Slope 1.4 jusec Fourier transform 11.7 msec Histogram 1.5 msec Classification 0.6 msec Matching two images 81.8 msec Resampling 0.05 msec

REQUEST SCHEDULE AND PROCESSING REQUIREMENTSN FILE WORLD MODEL EXTRACT SCHED. FOR NEXT OBSERVATION PERIOD ADJUST PORTION OF MODEL AND FORECAST OBSERVABLES NEXT PASS SCHEDULE AND WORLD MODEL SEGMENT

Figure 2.15.- IESIS uplink processing.

2.4.4 Sensor Requirements

IESIS sensor requirements are dictated by the specific terrestrial environment that must be scanned to fulfill an Earth resource mission. Full utilization of satellite capabilities and operating time demands useful sensing during daylight and nighttime passes of an orbit and during cloud cover.

Figure 2.16. - IESIS downlink processing.

Daylight operation involves observation by sunlight filtered through atmosphere twice before detection by satellite sensors. Most filtered sunlight lies in the visible region extending somewhat into the near-UV and further into the near-infrared. An optimum match between the daylight Earth environment and the satellite passive scanning system must include a visible, near-infrared, and some ultraviolet detection capability. A properly chosen 3-dimensional color space obtained using a red, green, and blue filter set yields color discrimination roughly comparable to that of the human eye. Detection at the chlorophyll absorption region near 650 nm gives useful discrimination for vegetation, while for water detection the low reflectivity region near 850 nm is useful (Schappell and Tietz, 1979). A pair of UV, five visible, and three near-IR bands should provide sufficiently broad color space (10 dimensions) to allow very widespread signature analysis of important terrestrial features such as crops, rivers, lakes, clouds, forests, and snow covers.

The nighttime environment may be scanned passively for thermal radiation at a temperature near 300 K. The black- body emissions of the cool Earth peak in the far-IR near 10 Aim. Four wideband far-IR sensors would allow accurate temperature and signature definition of nighttime features, although not to the same precision and resolution as with daytime sensing.

All-weather capability requires active microwave scanning of the Earth. The Synthetic Aperture Radar (SAR) operating at 1-10 GHz (Nagler and Sherry, 1978; OAST, 1980) is capable of essentially all-weather observation at good resolution. The SAR system also would augment nighttime passive measurement in the far-IR.

Altitude sensing provides useful information about terrestrial resources such as crop height, reservoir levels, or mountain snow cover. Height is recorded from differential altitude measurements performed at a boundary, e.g., by comparing the heights of crop tops to nearby level ground. A differential altitude measurement system is possible using rapid Q-switched LIDAR. Absolute altitude measurements can be obtained by LIDAR or microwave altimeter. Differential velocity measurements at a boundary (e.g., a river bank) can be taken by Doppler shift analysis.

Undoubtedly there will be requirements for additional specialized optical, infrared, and microwave sensor bands to detect important surface and atmospheric components such as ozone, water, water vapor, and carbon dioxide (Golovsko and Pakhomov, 1978). Somewhat artibrarily, six specialized sensor bands have been allocated to these uses for purposes of the present study.

Sensors are configured into a wide-angle medium- resolution sensor array and additionally into at least two arrays of narrow-angle, high-resolution sensors capable of independent accurate aiming over at least the swath width of the wider-angle array. Narrow arrays allow independent coverage of terrestrial features needed to satisfy conflicting requests during a particular orbital pass. They also may be used to obtain stereoscopic imagery, say, of cloud tops, by setting one array to view forward and the other aft during a pass and later combining the two image streams appropriately. Table 2.8 summarizes one possible sensor set.

2.4.5 Data Rate Estimation

The bit rate generated by one optical or IR sensor is given approximately by 8SBVW/A, where S is the number of sensor sets, ? is bands per sensor set, V is orbital velocity, W is swath width, and A is pixel resolution in km2 for an 8-bit pixel. For two sets of narrow sensors and one set of wide sensors the total bit rate generated at a 7 km/sec orbital speed is 8(2)(10X7 km/sec)(l 10 km)/(5X 10'3 km)2 + 8(1X20)(7 km/secX330 km)/(15X10~3 km)2 = 6.6X109 bits/sec. Of the remaining sensors the SAR will produce the maximum data rate by far. Today's SAR apparatus generates data at 0.65X109 bits/sec (Nagler and Sherry, 1978). Representative Doppler LIDAR, Doppler radar, and laser altimeters return data at the rate of several tens of kilobits per second. Thus, onboard computing capability requirements must be sufficient to handle data rates near 7X109 bits/sec. This is roughly an order of magnitude higher than that used in present Landsat orbiters.

2.4.6 Satellite Requirements

A summary of required measurement rates has been provided by Nagler and Sherry (1978) for a wide range of environmental and resource assessments. The necessary frequency of observation generally is lowest for land-based features, higher for ocean observation, and highest for atmospheric and weather assessments, with considerable overlap in the requirements. Table 2.9 indicates the frequencies of Earth observation and attendant swath widths believed reasonable for the IESIS system.

IMAGE Figure 2.17.- A projected advanced massively parallel processing system.

TABLE 2.8,- SENSOR CONFIGURATION OF AN EARTH OBSERVING SATELLITE.

Configuration and array of a possible set of active and passive sensors in an early mission. All sensor bands to be operated simultaneously if desirable.

Configuration

1 set of wide angle sensors comprising the full array of sensors to scan 330 km swath at 15 m resolution, or as limited by individual sensor.

2 sets of narrow angle UV, visible and near IR sensors, capable of accurate aiming, to cover 110 km swath at 5 m resolution; ~7X 10? bits/sec.

Sensor Array

10 bands - UV, visible and near IR (daylight) 4 bands - far IR (night) 6 specialized bands (atmospheric composition) SAR (all weather) LIDAR Differential height Differential velocity Altimeter

The IESIS satellite program is envisioned as developing in a long-term sequence carrying well into the next century. A detailed world model of land features already exists as contour maps covering a significant portion of the continents. Land features have sharp boundaries and vary only slowly over time. Oceans have wider geographic features that vary seasonally. The atmosphere requires three- dimensional modeling of rapidly varying phenomena. An obvious difference between land and ocean or atmosphere from a user standpoint is the large human population on land and its virtual absence elsewhere. Table 2.4 summarizes the characteristics of world models of the land, oceans, and atmosphere.

The logical deployment sequence of user-oriented resource satellites begins with a set of basic land-observing satellites whose world model already can be rather fully detailed. Since the satellites will spend about 75% of their time over the ocean it is natural to include ocean-sensing capability with as much ocean modeling as is feasible at the time of design and launch. Atmospheric sensing and rudimentary modeling should be included, both for understanding the state of the atmosphere and also as a necessary part of the interpretation process for sensor readings of land and ocean observations.

To assure long life for these sophisticated satellites, reasonably high orbits are required. Atmospheric path dis

TABLE 2.9,-POSTULATED OBSERVATION FREQUENCY AND SWATH WIDTHS.

Niche features observed Observation frequency, per day Maximum swath width, km Land 0.5 350 Ocean 3 700 Atmosphere 12 1400

tortion and sun angle introduce errors and complications into the interpretation process for imaging data. Path distortion causes reddening and other wavelength- dependent absorptions, and Rayleigh and Mie scattering are especially sensitive to particle size in the atmosphere and to sun angles.

The use of sun-synchronous satellites simplifies the situation considerably, a rational initial constraint which could be removed at some later time when more sophisticated modeling becomes available. It appears reasonable to have a set of sun-synchronous satellites operating continuously so that each Earth ground point is covered at the equator every 2 days by at least one satellite of the set. Sun- synchrony produces roughly the same sun-angle conditions over an observed land point for a particular satellite and helps to standardize image interpretation for that satellite at that point. An orbit near present-day Landsat altitudes (920 km, nominal) will support a long-lived satellite. If altitude is adjusted to a 14-1/8 rev/day rate, the ground trace of a particular satellite repeats every 8 days. Four such satellites could cover the Earth with the desired 2-day period.

The swath width required of a satellite for 8-day coverage at 14-1/8 rev/day is about 350 km. However, in order to take account of partial cloud cover the team proposes six satellites in sun-synchronous orbits. If these are placed substantially uniformly about the Earth's circumference the local viewing times for each satellite are spaced about 2 hr apart. Bunching may be desirable if there are reasons to pick a particular local viewing time. Polar conditions can be monitored by a seventh polar satellite, which may also act as a spare if one of the sun-synchronous satellites is disabled.

To relay data to the continental United States, two geostationary satellites are required. These satellites are also used to monitor global conditions, particularly cloud cover. Global cover information is compiled by IESIS to prepare each satellite for the tasks it can most usefully perform during its upcoming orbit, by enabling modifications in sensors and processing to optimize the information obtained from each series of observations.

Ocean coverage of a particular ocean point three times per day with a 700-km swath width requires 12 satellites, and atmospheric coverage at a rate of 12 times per day with a 1400 km swath requires 24 satellites, each with an 8-day repeat cycle (assuming the same sun-synchronous orbital parameters given above for land-observing satellites).

The technology available in the year 2000 (hypothetical IESIS deployment date) will, of course, dictate the actual satellite configuration employed. Still, the initial set of satellites should emphasize land observation with more sophisticated oceanic and atmospheric satellites phased in as the ability to model these systems develops. Figure 2.18 summarizes the basically land-observing system described earlier.

2.5 Time Phasing

? SUN SYNCHRONOUS ? 2-DAY PERIOD ? 6 SATELLITES-900 km, 99? INCLINATION ? 1 POLAR "SPARE" ? 2 GEOSYNCHRONOUS ? MODULAR ? SERVICEABLE

Figure 2.18. - Orbit characteristics: Possible initial IESIS satellite configuration.

The intelligent Earth-sensing information system proposed herein is an evolutionary system which considerably extends both planned and existing NASA missions. The time-phasing chart in figure 2.19 is oriented to development of the various components culminating in an operational system by the year 2000. Little attention was given to sensor development as this technology is driven by the various demands of other users and by general progress in this technical area (Breckenridge and Husson, 1979). Most attention was directed to software and artificial intelligence development as these lie at the heart of IESIS, although advanced hardware technology R&D also is required to achieve high packing densities, large wafers, fault-tolerant designs, advanced cooling techniques, advanced interconnections, more logic functions between array elements, advanced data output, and parallel input from sensors to buffer memory. Some of the major points are as follows:

- Research into automatic mapping and world model development should begin early. A world model for use onboard should be ground-demonstrated by mid- 1987 and a Shuttle demonstration of the world model/sensor operation completed by 1990 to meet the projected IESIS deployment date (2000 AD).

- Parallel processor development should be given high priority. The Massively Parallel Processor (a 128 X 128 array processor) will be operational in 1982. A 1000 X 1000 (or perhaps a 10,000 X 100) array for parallel processing should be developed by 1990 and should be flown on a Shuttle test satellite by 1995 to meet the 2000 AD deadline.

- Natural language user interfacing with the data system should be operational by 1990.

- Development of a model of the user population should begin immediately and be phased with natural language and world model development. The prospective ground demonstration of the world model using direct data from an advanced Landsat can be made available at some point to selected users on an experimental basis. The information on prospective selected users can form a preliminary user model.

- Signature analysis, data handling, and security will require continuing development and algorithm refinement. By 1995, software should be flown onboard both experimentally and as an initial phase-in on the autonomous satellites.

- A large world model encompassing terrestrial, oceano- graphic, and atmospheric components and a satellite system scheduler/controller should be ready by the year 2000.

- A gradual phase-out of Landsat D orbiters and phase-in of more autonomous "smart" satellites should begin. By the year 2000, fully autonomous satellites carrying world models should be available for long-term operation and initiation of the complete IESIS program.

LIBRARY DEVELOPMENT DATA ARCHIVAL SYSTEM SOFTWARE DEVELOPMENT SIGNATURE ANALYSIS, DATA HANDLING. SECURITY FINAL DESIGN OPERATIONAL , HARDWARE DEVELOPMENT IR, RF, OPTICAL AND TUNEABLE SENSORS, NAVIGATION. POINTING AND CONTROL. PARALLEL OUTPUT SENSOR MEMORY INTERFACE OPERATIONAL CAPABILITY _ INTERIM SATELLITE SYSTEM LANDSATS ON BOARD PROCESSING DEMO PROCESSING DEMO SATELLITE TO GROUND LOW RESOLUTION WORLD MODEL UPDATING USER MODEL DEVELOPMENT OPERATIONAL PRELIMINARY WORLD MODEL DEVELOPMENT TERRESTRIAL HIGH RESOLUTION ON GROUND LOW RESOLUTION ON BOARD Z * LOW RESOLUTION LOW RESOLUTION ON GROUND DEMO DEMO ON SHUTTLE PRELIMINARY OPERATIONAL NATURAL LANGUAGE USER INTERFACE PROCESSOR DEVELOPMENT 1000 x 1000 ARRAY ON BOARD 1000 * 1000 ARRAY ON GROUND 128 x 128 ARRAY ON GROUND

1995 1990 1985 2000 IESIS ATMOSPHERIC MODEL OCEANOGRAPHIC WORLD MODEL TERRESTRIAL WORLD MODEL SATELLITE SYSTEM SYSTEM ASSESSMENT OCEANOGRAPHIC AND ATMOSPHERIC 2020 IESIS Figure 2.19IESIS time phasing chart, 1980-2020. 2.6 References

Breckenridge, R. A.; and Husson, C.: Smart Sensors in Spacecraft: Impacts and Trends, in Remote Sensing of Earth from Space: Role of "Smart Sensors," Roger A. Breckenridge and Martin Summerfield, eds., Progress in Astronautics and Aeronautics, AIAA, vol. 67, pp. 3-12, 1979.

Chien, R. ?.; and Peterson, L. J.: Image Compression and Reconstruction Using Feature Extraction, in Proceedings of the 5th International Joint Conference on Artificial Intelligence, MIT, 22-25 August 1977, Cambridge, p. 658, 1977.

Gilmore, P. A.; Batcher, ?. E.; Davis, M. H.; Lott, R. W.; and Buckley, J. ?.: Massively Parallel Processor. Phase I Final Report, prepared by Goodyear Aerospace Corporation for Goddard Space Flight Center. July 1979.

Golovko, V. A.; and Pakhomov, L. A.: Simultaneous Determination of Atmospheric Temperature, Water Vapor Profiles and Thickness from Infrared Interferometer Measurements on Board "Meteor 28." In Remote Sounding of the Atmosphere from Space (Advances in Space Exploration, vol. 4), COSPAR, Pergamon Press, New York,pp.247-250,1978.

Graham, Donald Norman: Image Transmission by Two- Dimensional Contour Coding, Proc. IEEE, vol. 55, no. 3, pp. 336-346, March 1967.

Nagler, R. G.; and Sherry, E. J.: Global Services Satellite Circa 1995: Volume II ? Environmental and Resource Management Measurement Services, Jet Propulsion Laboratory, Pasadena, California, 30 September 1978.

Office of Aeronautics and Space Technology (OAST): NASA Space Systems Technology Model: Volume I - NASA System/Program Descriptions and Technology Needs, NASA TM-81041, May 1980.

Outlook for Space: NASA SP-386, 1976.

Rich, E.: Building and Exploiting User Models. In Proceedings of the 6th International Joint Conference on Artificial Intelligence, 20-23 August 1979, Tokyo, pp. 720-722, 1979.

Schaefer. D. H.: NASA End-to-End Data System Massively Parallel Processor, Goddard Space Flight Center, Greenbelt, Maryland, 30 May 1980.

Schappell, R. ?.; and Tietz, J. C.: Landmark Identification and Tracking Experiments. In Remote Sensing of Earth from Space: Role of "Smart" Sensors, Roger A. Breckenridge and Martin Summerfield, eds., Progress in Astronautics and Aeronautics, AIAA. vol. 67, pp.134-151, 1979. N83-15350