GEOG 583
Geospatial System Analysis and Design

Reading Assignment

PrintPrint

Read

What is a Software Architecture? by Peter Eeles of IBM. This will give you a high-level, authoritative view of what a software architecture consists of.

Think About

What parallels do you see between the designs of software architectures and the other design activities we have covered so far in this course? In a lot of ways we are covering the major pieces of an overall "structure" of design in this course.

Read

ESRI guide to the ArcGIS Software Architecture, pages 1 through 6.

Respond

This is a good summation of how ESRI conceives their system architecture. It's aimed at ArcObjects developers, so don't worry about programming details at this point. Just try to get a feel for how the pieces fit together, and in particular, try to think of how the definitions of software architecture from the first reading play out in the context of ArcGIS (or other GIS systems). Respond to the readings by posting your take on how the second reading (on ESRI's architecture) fits into the categories in the first reading (on software architecture). What is missing from the ESRI document that the IBM article mentions? To get credit for this assignment, you must post at least one original comment and respond to at least one comment by your fellow students. I encourage you to include links to other content that bolsters your argument.

Note

To post a comment, scroll down to the text box under "Post new comment" and begin typing in the text box, or you can choose to reply to an existing thread. When you are finished typing, click on either the "preview" or "save" button ("save" will actually submit your comment). Once your comment is posted, you will be able to edit or delete it as needed. In addition, you will be able to reply to other posts at any time.

Don't see the "Post New Comment" area below?  You need to be logged in to this site first. Do so by using the link at the top of the menu bar. Once you have logged in, you may need to refresh the page in order to see the comment area below.

 

Comments

wmh134's picture
Submitted by wmh134 on

The Esri software architecture document generally defines structure as described in the IBM article, categorizing components broadly by base services, data access, map analysis, and map presentation. Those four categories are in turn used in all three ArcGIS product families: Engine, Server, and Desktop. But unless I'm missing key details on account of my lack of technical knowledge, it does not seem to address behavior, or the interactions between structural elements. I must admit to not being entirely certain about that, however, given the IBM article's point that architecture focuses only on significant structure and behavior, elements that have a long and lasting effect. I'm not sure whether I'm simply overlooking a high-level conceptualization of behavior in the ArcGIS architecture document, even after numerous readings.

The Esri document's inclusion of compatibility between ArcObjects 9 and 8.3 as one of four key concepts struck me as an example of a way in which its architecture balances stakeholder needs. In this case, the needs of the customer or external developer. Not mentioned is the balancing of internal stakeholder needs between, say, project managers and the marketing department. Compatibility and the other three key concepts of the ArcGIS architecture — modularity, scalability, and multiple platform support — together form a kind of rationale for, as the IBM article puts it, why it is the way it is.

As Dr. Robinson said in his email introduction to this week's lesson, I'm also really looking forward to hearing the perspective of those of you from a software development background. Even the IBM article's outline of what a software architecture actually is was conceptually tough sledding for me, and the ArcGIS-specific description in the Esri document only more so.

ezg135's picture
Submitted by ezg135 on

I am a little lost in this lesson since I do not know much about software and how to develop code.  It thought I understood it when I took Fortran many years ago, but now it is all about classes, objects within the classes, instances … and that is when I got lost! I tried to learn Java, but did not like it at all, may be because I did not understand it, and did not made sense to me.  I had my doubts about the behavior of the ArcGIs architecture too.  I do not recall reading how the three products interact, but looking at the diagrams on the left part of the pages I saw that things like Map Presentation, Map Analysis, and Data Process are included in the three diagrams.  I assumed then, that if they share this elements (ArcObjects I guess) there has to be interaction between the three product families, and therefore a specific behavior.  This and the fact that ArcObjects are the components that make up ArcGIS.  What it is not explained is in what way all these ArcObjects interact with each other.  It looks like they interact on a high level, but this is just my thought.

But again, it does not mean I am right since my lack of knowledge and experience in this field

Elena.

mzd205's picture
Submitted by mzd205 on

Mark and Elena,

I am with you on this set of readings - it took getting used to for someone without a deep technical background. I did take Java programming over 10 years ago in high school - it was Java, so the OOP language came back to me as far as the most rudimentary understanding of objects and their data and methods.

I agree that the ESRI document falls just short of providing a coherent rationale of how all the major elements interact with each other in their role as pieces of a system. While functionality is addressed it is a more wide-ranging overview without going into the logical basis of how one ArcObject depends on another - and how data runs through the feedback loops between the different objects. I would therefore hesitate to call this document an architecture just based on the limited introduction to the idea we've had so far.

jln240's picture
Submitted by jln240 on

I think you got the main point of the diagrams in a way that they are all built on the same basic structure.  This is all based on the fundamental blocks of ArcGIS... ArcObjects.  Esri actually has a well documented resource on ArcObjects because that is how developers create custom tools... reading the Object Model Diagrams (OMD), which is the graphic representation of how all of the tools are connected and associated, a UML.  This will give you a quick introduction to OMDs but they can get pretty gnarly and confusing that it takes an entire semester of Geog 485 to even get used to reading them. 

hxw184's picture
Submitted by hxw184 on
In the context of the IBM article, I would say that architectural "style" of the ArcGIS architecture equates with Object-Oriented Programming. ArcObjects is a collection of software objects or components, organized into libraries. Software objects are described and compared to real world objects in the following link: http://docs.oracle.com/javase/tutorial/java/concepts/object.html
 
Real-world objects share two characteristics: They all have state and behavior. Dogs have state (name, color, breed, hungry) and behavior (barking, fetching, wagging tail). Bicycles also have state (current gear, current pedal cadence, current speed) and behavior (changing gear, changing pedal cadence, applying brakes)... Software objects are conceptually similar to real-world objects: they too consist of state and related behavior. An object stores its state in fields (variables in some programming languages) and exposes its behavior through methods (functions in some programming languages). Methods operate on an object's internal state and serve as the primary mechanism for object-to-object communication. (Oracle Java documentation)
 
This OOP framework "defines the structure" and "defines the behavior" of the components in the system. (I'm going to have a go at this, hopefully I don't get it wrong!) A geometry object might be of a type "point", which has properties such as spatial reference and x,y coordinates and might have a "project" method that transforms it to a different coordinate system.
 
So far, this speaks to the structure and behavior part of the architecture. I think that environmental factors would be the base programming languages used to write the objects (their limitations and strengths would tend to influence the design), the operating system differences and "threading" (which I don't understand, but seems both hardware and software related). Other aspects of the IBM article show fewer direct parallels. As far as focusing on significant elements, I don't see it, except possibly in the extendability of the framework - that they've focused on core libraries? As Mark stated, the rationale piece is somewhat addressed by the four priorities listed in the chapter: modularity, scalability, multiple platform support and compatibility. I think that's where the balancing stakeholder needs comes in, too. The text seems to suggest that there were some sacrifices of compatibility in favor of modularity.
 
I found the IBM article dense material, mostly because of the volume of definitions! It left me a bit confused about what can be called a software - or system - or enterprise, architecture.

rcl164's picture
Submitted by rcl164 on

I think you pointed out a very interesting component of this discussion on software architecture; how to balance the needs of the stakeholders.  I feel that the readings glossed over this concept.  Even though it appeared like a difficult task, the authors did not elaborate on how this process works.  I wondered if there was some sort of prototyping process that happens or if there are industry standards that serve as templates.  I researched this a bit more and found a White paper that describes the process as a combination of both.  It seems that the industry has well developed “viewpoints”, similar to what we might have defined as “personas” in our earlier lessons, and that based on specific viewpoints, there are templates that the industry uses: 

“A viewpoint is a collection of patterns, templates, and conventions for constructing one type of view. It defines the stakeholders whose concerns are reflected in the viewpoint and the guidelines, principles, and template models for constructing its views. Architectural viewpoints provide a framework for capturing reusable architectural knowledge that can be used to guide the creation of a particular type of (partial) AD.”

The white paper also makes it clear that communication with the stakeholder groups is essential, and depending on their role, they should be provided with paper prototypes that can help guide the discussions and draw attention to problem areas.  In some cases more informal “pictorials” are used for this process (slide 26).   I was pleased to see that the white paper also pointed out that there are pitfalls associated with the “viewpoint” method, including selecting the wrong set of views!  

 

acr181's picture
Submitted by acr181 on

A version of this class taught long ago used to spend a bunch of time on making UML diagrams, which can help add a visual structure to these rather abstract collections of software elements. We moved on from that after a great deal of feedback from students that it was virtually never used in professional practice. Having said that, UML definitely provides a structured method for visualizing the connections and behaviors between objects in programs. This is a pretty nice guide that walks through some basic real-world comparisons to explain how UML works, and here's a nice little web tool for making your own UML diagrams right in the browser.

The only way I've ever seen UML tools used has been when developers want to mock up the software architecture. I will add that I've never seen anyone actually follow the "rules" of UML very strictly, but then again I've never worked in the type of big software enterprise that UML is really designed to suit.

jls1082's picture
Submitted by jls1082 on

I found it is interesting to see how ESRI designed the architecture of their software for the purpose of creating an environment that makes it easier for their customers to create customized systems that will serve the needs of their organization and end users.  It displays the ripple effect of how the design of one software system can positively affect the design of another.

The IBM article emphasizes understanding and establishing the components of system architecture and how those components interact with each other and the environment in which the system will function.  The ESRI article describes how their software is scalable (can be used in many different types of applications) and compatible with the ArcGIS system across versions so that external developers don’t have to change their code in order to work with the latest software release.  This seems to align with the IBM notion of considering your stakeholders and the interaction of the software with other system environments.  Since ESRI tried to minimize negative effects on their stakeholders and accounted for wide application use of their software when they designed the updated system, it seems they recognized the constraints and mission of their users and the environment in which their product will be used.  ESRI even addressed the internal and external constraints of their customers when the article explains in the API section that choosing the correct API for the developer’s application depends on the products they use, the end user’s desired functionality, and the developer’s programming language experience.

The ESRI article did not really touch on their own internal constraints during the design process, such as why in the Geometry library the geometry primitives are not meant to be extended by developers when most everything else can be extended.  Is this due to some limitation in the ESRI system or did they just not think it is necessary? 

I don't have any experience in software design, but from what I understand, it seems as though ESRI is very aware of how their software interacts with external applications and previous versions of their products and how this impacts their users.

- Jennifer

mtd206's picture
Submitted by mtd206 on

I think ESRI really has to make any changes they make with compatability in mind. One of the small pains that comes along with being the dominant software in a field is that other people design extensions and add-ons to it to do what they need to do (I'm thinking in particular here of the DoD realm). If ESRI broke those connections completely, there would be a LOT of unhappy customers.

acr181's picture
Submitted by acr181 on

You're right that they've gone to great lengths to design a flexible architecture, Jennifer. My understanding is that a huge part of Esri's core business is actually in the Professional Services division, where they work with all sorts of end-users to customize solutions for their needs. You can get a sense of all of their major customization markets by checking out the list that appears when you look at their industry areas here.

This is an important thing to keep in mind as we talk next week about FOSS4G solutions and how they are positioned in the market. I think you'll see that companies in that realm are making a very similar argument to attract customers (we have flexible tools from which we can make you a customized solution).

jls1082's picture
Submitted by jls1082 on

ESRI seems to have customer service at the forefront of their business, which any good business should. I didn't realize they provided such complete planning services.  I can see this being a huge bonus for any organization, but especially for someplace small that might not have the staff capable of designing a new system or researching all the possibilities.  It's a one stop shop for the organization and a win-win for ESRI since they are able to sell more product.

ezg135's picture
Submitted by ezg135 on

 

In the first reading, the commonalities in all the architecture definitions are that an architecture is concerned with structure and behavior, with significant decisions and not with fine-grained details, it has a particular kind of pattern that defines the style, it is influenced by stakeholders and its environment, and its build on rational decisions.

In the second reading it is shown the structure and behavior of the software architecture.  There are three different diagrams showing the structural aspects of the three product families, and how they interact: ArcGIS Engine, ArcGIS Server, and ArcGIS Desktop.  Many of the ArcObjects components are used within all three families.  Besides the diagrams, there is a description of each one of the three products explaining how they interact with the operating system, Internet, and the interaction with the end users.

Software architecture focuses on significant elements.  What is important about this fact is that if the architecture has relative stability, even when the set of significant elements changes, the architecture still will be stable.  There are four key concepts that resume the main changes made in ArcGIS 9: Modularity, Scalability, Multiple Platform Support, and Compatibility.  A good example of architecture stability is described in the Modularity section.  It explains the benefits of using DLLs.  While ArcGIS 8.3 had all the ArcObjects components in one large block of functionalities, ArcGIS 9 is divided into a number of libraries.  These changes can only be made if the architecture of ArcGIS is very stable.

It is obvious that the ArcGIS architecture and the changes made are based on rational decisions.  All over the second reading it is explained why the changes are made, and what are the new advantages in doing so.  Here is two examples of were the rational thinking is shown:

  • The use of DLLs provides a flexible system at minimum memory requirements, but if we have too many files it will affect performance.  So the components sharing the same set of requirements were effectively packed into DLLS.
  • Scalability is important because it is the way to ensure that the server can handle many users, and a good way to achieve this is using “Threads in Isolation.”  It ensures that the architecture is being used efficiently.

In an architecture, a crucial thing of the decisions based on rationale is to document all these decisions and why they were made.

The architectural style can be thought as a particular sort of pattern.  A good architecture will be well documented, this make it easier to reuse some part of it which is a good practice.  In the case of ArcGIS this was useful to maintain the compatibility of ArcGIS 8 and ArcGIS 9, although some changes in the architectures were made.  In this case, not only the developers could reuse some parts of the old architecture, but it was important that things like external developers not having to change their code only because of a new release of ArcGIS.

When it comes to the architecture environment ESRI guide focuses mainly on the external technical constraints.  The system definitions by IEEE 12207 and RUP SE include the concepts of software, hardware, and people.  But in the second reading the hardware description with its requirements and constraints is not included.  The developers are mentioned, but the team structure is not known neither are the skills required for these developers who made all the changes mentioned in the ESRI guide.  On the other hand, it is specified more than once than the language chosen to develop ArcObjects was C++, Microsoft Component Object Model was used, and how it work on different operating systems.  The scope is centered on software architecture, and in some information architecture, not mentioning neither hardware nor organizational architecture.

Some stakeholders can be inferred, like the corporation, ESRI.  The team of developers being also the maintainers, and the final users are only mentioned in those six pages, not finding a description of them.  System administrators, marketers, clients and project managers are not taken into account in the reading.

Elena.

 

frg107's picture
Submitted by frg107 on
There is a similarity between traditional architect design and software design. In this lesson's reading, I found the importance of the basic unit of design very interesting. Similarly to traditional architecture, software design encounters basic building blocks that indicate the limits of our design. Depending on the level of detail, we can find different building blocks. For example, if we want to focus on a molecular scale, we can compare the sand grains that are part of a brick, with the "bit" on our programs. We can be as detail as we want. Likewise, we can use the DLL as a basic unit of design to organize all the procedures in libraries, and depending on our choice, we can have different resulting structures and consequently different architecture styles. The importance of modularity, as well as scalability, is well established in architectural designs. Similarly, software architecture will embed in the design the structural support and compromise to allow the system to be compatible, to grow and to adapt to a range of changing variables.

elk186's picture
Submitted by elk186 on

The IBM article lists the distinct conditions that Eeles believes surround software architecture. He claims that an architecture:

  1. defines structure

  2. defines behavior

  3. focuses on significant elements

  4. balances stakeholder needs

  5. embodies decisions based on rationale

  6. may conform to an architectural style

  7. is influenced by its environment

  8. influences team structure

  9. is present in every system

  10. has a particular scope


For the most part, I can see how Esri’s ArcGIS software architecture achieves these various conditions:

        1. Esri’s document does discuss the structure of ArcGIS’s architecture as it relates to ArcObjects. It also goes into detail on how these Objects relate to each other, especially between the Engine, Server, and Desktop “product families”, as Elena notes above in a comment on Mark’s post.

        2. Mark’s post indicates that he is not sold on Esri’s coverage of the behavior definition, but I believe this is where the “Modularity” and “Scalability” concepts of Esri’s architecture fit in. These concepts describe the core behaviors of the new ArcGIS 9 architecture, while going into great detail on why.

        3. In this sense, while Esri does focus on the significant elements of the architecture, this specific document does go into a bit too much detail on the nitty gritty, but I think that is a result of the intended audience of this document as opposed to the focus of the architecture as a whole.

        5, 7 and 9. Esri’s documentation does incorporate the rationale behind certain decisions, aiding in the understanding of these decisions. This documentation also serves to show how Esri accounts for maintainability and best practices, as they illustrate with their discussion of performance and DLLs in the Modularity section and the discussion of threading in the Scalability section. The rationale given for why Esri made these choices make evident the environment in which they expect and require their architecture to operate.

        8. ArcGIS 9’s division into Engine, Server, and Desktop lines could very well echo the divisions of their teams. Not being present within Esri’s organization around 2004 when version 9 was released prohibits me from knowing whether these divisions existed in some way before version 9 or if this new architecture even resulted in new teams based upon these divisions after the release of version 9. I echo Eeles’ concern that we may not know whether architecture influenced team structure or the other way around.


There are a couple of conditions I don’t believe Esri’s document adequately addresses.

        4. Esri’s document fails to address most, if not all, of Eeles’ stakeholders needs. Their compatibility section expresses great concern for external developers, but a small side note on pg 26 really expresses their lack of care for these developers, who they tell to basically redo all their code they developed with ArcGIS 8 to fit into the new ArcGIS 9 architecture which they decided to change. Additionally, the system administrator, marketer, customer, project manager, and end user are neglected by this document, in my view.

        6. The concept of an architectural style from the IBM article left me wondering if there are specific architectural styles that work best for, or have been explicitly developed for, GIS and spatial data. Esri’s document did not answer this question for me, as it is unclear how their architecture fits into wide GIS developments. While the modularity they discuss does imply a reusability, in a sense, I doubt whether this “reusable parts” paradigm is really what is at the base of Eeles’ discussion of architectural styles. I think there is more to it, an impact to a broader notion and not just reusability within the architecture itself.

        10. I think that a large number of us are likely familiar with Esri and how their software architecture fits into the hardware architecture and broader system and eneterprise architectures, but I don’t think this document explains this. Other aspects of their corporate architecture aren’t mentioned, leaving out how this works in a broader context. I will give them credit for taking hardware considerations into account (see the mention on pg 24 thin/thick clients and server), which Eeles indicates should not be ignored.

 

Overall, ESRI's architecture fits well into the categories of the IBM reading on software architecture, but doesn’t discuss the broader scope of their software architecture, nor where it conforms to an architectural style, and additionally neglects a number of stakeholder concerns.

jkl178's picture
Submitted by jkl178 on

Regarding #4 - I don't know. To me, it's just the nature of technology that's constantly evolving and adapating to new practices. I wouldn't expect any oragnization to support a legacy system forever (especially an external product), so I think the burden does fall on the developer to keep their product working. What Esri mentions in its guide is that it minimizes the amount of things developers have to change with each release. I'm sure a developer sees it differently though :P

elk186's picture
Submitted by elk186 on

Good point, Jonathan. I guess their side-note on page 26 that came off to me as "redo all your code!!!! haha!!!" didn't sit very well. It makes a lot of sense that tools will change (hopefully for the better) and it's important to inform the people using those tools or developing from those tools about how these changes will effect their work. If big companies didn't change their products, especially ones that a large number of people rely on, those products would be stuck as they are and couldn't get any better. Like if Apple had to keep all of it's iPod/iPhone chargers as the former iteration and wasn't able to change to the Lightning cables. Yes, people wouldn't have had to buy new cables, but the technology wouldn't necesssarily have advanced to a newer, better cable that isn't direction specific and is easier to use. Thanks for the food for thought :)

hxw184's picture
Submitted by hxw184 on

I wonder how many people who ported all their code to 9.x had to scrap or rewrite it for 10.x and then again for 10.1, etc.? I'm not sure how much the architecture changed between all those versions, but it is still daunting to think about. Here are instructions for porting from 9.3 to 10.0, and some changes in classes from 10.1 to 10.2. The 10.x ->10.1 was a big version change for the desktop and server software, but maybe they kept with the modular approach described in this article. It would make sense that upgrading would be easier without the mother of all libraries to package, but still, the fact that there are all these changes to be accounted for must be just part of the job for developers! I know even the small amount of Python I've worked with in ArcGIS changed with the implementation of the arcpy module, and the change from Python 2.x to 3.x will break some stuff as well. But as you say, it is all hopefully for the best in the end and will at least reward a developer who documents, modularizes and employs other best practices.

jls1082's picture
Submitted by jls1082 on

My organization just upgraded to ArcGIS 10.3, which issues the release of Python 3.4.  When I tried to run a code that I created using python 2.7, I got an error saying that it does not recognize the arcpy module.  I found this ESRI article describing the conversion to Python 3 and what you can do to accommodate the changes which includes an automatic program you can run to convert your code from 2.x to 3.x (apparently it is 95% effective and would probably require some manual tinkering, but that doesn't sound too bad).  I haven't delved into fixing my code yet, but at least there is some support to assist with the change.  I also found an article from Python that explains why the transition was made from python 2.x to 3.x and how the system has evolved.

hxw184's picture
Submitted by hxw184 on

Thanks so much for these informative links. I am looking to upgrade soon and these are both very helpful. Bookmarked!

jkl178's picture
Submitted by jkl178 on

LOL that reaction was perfect. I totally see how that sidenote could come off that way. And I'm sure many developers actually feel that way too

jln240's picture
Submitted by jln240 on

I don't know.. I think, if I was a developer I would see that as one, a challenge and two, job security... then again, I might also be looking at that from the perspective of a contractor working for the federal government who needs to constantly prove that we are needed.

acr181's picture
Submitted by acr181 on

This legacy issue is a huge one, isn't it? If you stop and think about it, what would happen if Esri decided to no longer support shapefiles in future tools? There are tons of downsides to shapefiles, and lots of competing formats that eliminate all of those downsides. But everyone's got data in that format, so it keeps on chugging away.

My guess is that the recent development of ArcGIS Pro signals a departure from a lot of legacy code. We know for sure that they have finally gone to 64 bit with it, and that the truly awful screen renderer was finally updated to the 21st century so that your maps won't look like jagged garbage on screen anymore. I bet, though, that with every new tool they port over, it's someone's job to make absolutely sure that giant client XYZ can still use some weird legacy data format and process layers the way they've always done it.

I've recently started to work on ways to archive the music I record for similar reasons. I have multitrack recording projects which link lots of information about mixing/editing to raw .WAV files. The raw recorded tracks will always be easy to open in a wide range of tools. The .wav format is like a .jpg at this point - it isn't going anywhere. But the software used to edit and mix is the really tricky part - I have some projects that are more than a decade old, so I've begun importing them into a new piece of software that's more popular than the one I've always used, and then saving those projects in multiple formats from that new application.

Imagine if we were forced to do that with GIS - one candidate for replacing the shapefile that is gaining a lot of traction is the GeoPackage format, which is backed by a lot of proprietary and open source characters. And while that would advance data interoperability, what of the project files like the ones I'm worried about in my recording software. The .MXD's of the world are much more problematic, as they are really not easy to exchange into other formats that would work nicely on another platform (let's say you want to migrate from ArcGIS to QGIS - you won't find a perfect MXD conversion method to my knowledge - would *love* to be proven wrong here though so please inform me if you know of a way!).

jln240's picture
Submitted by jln240 on

When I first saw the ArcGIS Pro demo this week, I was thinking in my head, "What would we need ArcGIS Desktop, ArcGlobe and ArcScene for then if this can do things way better?".  They were making it all out to be the next big thing but then apparently they havent really rolled out all the tools yet and they do not intend to replace ArcMap, which I am not entirely sure why.

elk186's picture
Submitted by elk186 on

Jeanne, I got the same feeling from the ArcGIS Pro demo I saw this week (were you at the Fed Users Conference, too?). It doesn't seem like they are making it to replace ArcGIS or the legacy formats people have been discussing. Rather, it is their way of making an impact on the market that is moving beyond these legacy formats-- getting their feet in *all* of the markets. I can see them continuing to support all those other tools until slow organizations (aka the government) have time to catch up with things.

mtd206's picture
Submitted by mtd206 on

Regarding your #10, ESRI may claim to take hardware considerations into account, but I haven't seen much evidence of this in the real world. We have both "thin" (cloud/server-based "desktops") and "thick" (actual at-the-desk desktops) in my workplace, and we can't even think of running ArcMap on the thin clients. All of my experience with Arc programs would seem to indicate that running "lite" is about the last thing on ESRI's mind - even if I'm only messing around with vector files.

jmv233's picture
Submitted by jmv233 on

I tend to agree with your point that the ESRI doc doesn't necessarily adress all stakeholder needs, and I did see below the points about how it is probably covered in other chapters in the documentation.  I'm wondering, though, who the stakeholders are for a topic like ArcObjects.  Developers - sure.  Administrators - yes, they have a vested interest in knowing how the product works.  Beyond that?  I'm not sure that the PM or end user are stakeholders from this perspective.  Stakeholders, like architecture, have a scope.  The end user of the software built using ArcObjects has a stake in the software built using ArcObjects, but would ESRI consider them stakeholders in its own architecture?  Definitely a grey area, I would say.

hxw184's picture
Submitted by hxw184 on

This is a nice distinction, Jen. I would agree that the main stakeholders in this document are the developers. Not all documentation is written for all audiences, and it's pretty evident that the documentation is aimed solidly at those who are, in effect, the ArcObjects "end-users".

mzd205's picture
Submitted by mzd205 on

The Eeles article makes clear that architecture has a variety definitions with common elements prominently featured in most. Eeles then delves into a discussion of the common themes – in particular he emphasizes that many definitions of architecture include

elements of a system

the key relationships among the elements

the defining properties of each key element

In addition to this functional core, the context within which it resides – the “environment” (including mission, technical and regulatory constraints) and the need to tailor the scope to stakeholder needs – is deemed critical to the description of what constitutes an architecture.

The ESRI Developer Guide portion on ArcGIS architecture address most of these points but not in a coherent manner – I believe this is intentionally so, because their audience is prospective clients and their detail-oriented developers who want to digest the technical data more so than see the big-picture architectural design.

 

Eeles gives an example of Classes (as in Object-oriented Programming) when presenting to the reader UML diagrams as his version of structural elements that comprise an architecture. In the ESRI paper, the major elements are described on a more conceptual level – the interrelated “product families” of ArcGIS Engine, Server, and Desktop.

Eeles emphasizes relationships among the elements (Classes in the UML example) – and how instances of each object and their dependencies work together to define the behavior of the system as a whole. The ArcGIS paper on the other hand delves little into the logical relationships among the components in each product - e.g. Base Services and Data Access in ArcEngine- but instead presents an overview to the reader highlighting “commonality of function” and flexibility in implementation within the product family – their modularity, scalability, backwards compatibility from 9 to 8.3, and multiplatform support. Even with a lot of very technical content, it reads more like a White Paper or an explicit advertisement rather than a systematic description of the ArcGIS core elements, relationships, and defining properties.

The ESRI paper does a better job defining the system’s overall technical constraints, following Eeles’ emphasis on defining environment, scope, and stakeholder needs when describing an architecture. The system’s constraints as relating to extension limitations depending on choice of API, and multiserver functionality are well defined on pages 26-27.

wmh134's picture
Submitted by wmh134 on

I think you better articulated my thoughts on the Esri software architecture document than I managed to do in my own comment. Specifically, that the Esri document is a more conceptual description of a software architecture than the IBM article's point-by-point definition. And even more specifically, that the Esri document at times read a bit more like marketing material than a technical description of a software architecture. That said, I skimmed the rest of the document past the section we read for this week's lesson, and it appeared to be more along the lines of a systematic description of ArcGIS structure and behavior.

elk186's picture
Submitted by elk186 on

Your comment about the coherency of the Esri document got me thinking- Esri called this chapter "ArcGIS Software Architecture" in a book entitled "ArcGIS Engine Developer Guide". Their audience for this document, as both you and I discuss in our comments, is very much the developer community, specifically developers using ArcGIS Engine. In this sense, their intent, like you say, is to provide technical information to "detail-oriented developers". These developers likely don't care as much about the aspects of the architecture that Eeles stresses in his article. So while this chapter in-and-of-itself might not hit on all of the principles that Eeles discusses, that doesn't mean that Esri didn't consider these or document them somewhere else.

This thought motivated me to search for other places where Esri may have documented their software architecture, without focusing just on developer stakeholders. I found this document, which does seem to address more of Eeles' principles. This document seems more like Eeles' idea of a documentation of rationale, to be used later in reevaluating those parts of the system or to justify certain choices, as well as covering many of the significant elements of the architecture and updating it numerous times over the years as things have changed. I didn't read all of this lengthy tome, rather based these notions on the way the document is framed in the table of contents. Now, whether this document is really intended to be read by anyone, or is just available in case Esri needs to say "see, this is why we did such and such" is unclear, but Eeles and I agree on the need to have such a document in case that need arises!

jkl178's picture
Submitted by jkl178 on

First of all, technical writers are great because they know their stuff. But because they know so much, it's probably harder for them to break down a complex system into simpler terms... I would've expected a "guide" to be a little more guide-like. The IBM article was helpful on that front.

To my understanding, Esri's architecture also fits into Eeles' discussion about architectural style, which is defined as component patterns that can be reused. These modular components in ArcGIS - base services, data access, map analysis, and map presentation - allow for code re-use among the three product families.

The products are also an example of how an environment affects architecture. The primary environment in which Esri has chosen to operate (Windows) makes it difficult to operate on other platforms (eg. Macs). The solution to this platform support seems to be focused on ArcGIS Online (sub-product of ArcGIS Server??).

atc147's picture
Submitted by atc147 on

I think you're right - ArcGIS Online seems to be ESRI's primary solution to ArcGIS' lack of compatibility on operating systems outside of Windows, and while they've made many improvements to the platform and service over the past two years, it still lacks many of the features and capabilities of ArcGIS, which is disappointing. I've heard there is a fairly simple way to run ArcGIS on a Mac - by partitioning the OS using software called Bootcamp or VMWare (see this article by the University of Illinois) but I have yet to try it myself.

jkl178's picture
Submitted by jkl178 on

Yup - I switched over to a Mac late last year. I use Parallels, and ArcGIS runs decently. Haven't had it crash yet. I've heard VMWare works well too. But still, a little inconvenient for those who don't want to get the extra software just to run 1 program. I wouldn't have made the switch if those options weren't available. (I think there's a student discount, so take advantage of it while you can)

mdc305's picture
Submitted by mdc305 on

I'm glad to hear about more folks using the Mac to handle their GIS workflows.  I've overheard several conversations over the last few years about running Windows software on a Mac, both with VMWare and Parallels.  The general consensus is that the experience has been positive, both for GIS use and more generalized purposes.

For most of the folks in that dual operating systems boat, it would seem that price is less of a consideration, as it would make the most sense to carry a single machine with the capability of running both operating systems.  But, there's also an argument for the price of a complete Windows laptop versus the standalone cost of the Windows O/S as an add-on to the Mac.  Since the cost of a Mac is somewhat higher, I think that the brand loyalty comes into play here, and that having Windows applications on board would just be an added bonus.

Who knows what the computing future will hold?  As more folks go the Parallels route, maybe ESRI will find a market for a fully functioning copy of ArcMap that runs within the native Mac environment.

jln240's picture
Submitted by jln240 on

I guess your parallels solution is better than the one my classmate from GIS Summer School (5 years ago) in UCR was using.  I remembered a lot of unfriendly words coming out of him because of his insistence to use his Mac than the ones provided at the lab.  That I think is Apple loyalty.  He never buckled.  Anyway, I saw a lot of presenters at the conference and summit today were using Macs and oddly enough, they were the ones who didn't have technical difficulty in presenting their live demos.

atc147's picture
Submitted by atc147 on

The ESRI article, aimed at developers, explains the updates and changes that have been made to the ESRI system architecture in ArcGIS 9 compared to previous 8.x versions. With my very limited understanding of developer-speak, it seems like what ESRI is describing falls into the categories of both “information architecture” and “software architecture” as identified by the IBM article. While the ESRI article clearly articulates the impact that the changes to ArcObject components will have on modularity, scalability, multiple-system support, and compatibility, what the article seemingly fails to mention is the behavioral impact that these software changes will have on the other elements of the system architecture, including impact on hardware and the organization (and people) utilizing the system. That being said, ESRI does emphasize the prioritization of a “unified software architecture” so that user investment in these changes can be realized across the different ArcGIS products - but even still (and perhaps much of this is obvious to those who are developers), aside from the “check your code” flag at the end of the article, there seems to be a lack of discussion on relevant hardware/resource needs that might be impacted by the software update.


This concept and reading was tough-going for me, but reading all of your posts is helping me to better understand it all, so thank you!

prr123's picture
Submitted by prr123 on

This introduction to the relatively new discipline of software architecture is the first of a four-part series on "architecting" in general. The author begins by defining the discipline's key terms and goes on to explore what a well-designed architecture contributes to the environment in which it is deployed.

I believe the statement above differentiates the IBM article from the ESRI article in the sense that Eeles explains architecture in a broader more basic sense than ESRI. The ESRI article (chapter) to me targets both developers and perspective clients looking to purchase or upgrade their current ESRI software. All the tech talk was way over my head for understanding, not to say that I was not aware of C++, DLLs, etc and the broad spectrum of hardware associated with computer science and/or GIS, all that is just not anything I have real experience with other than end user.

While reading the IBM article, I had a much better understanding since it seemed to be "dumbed down" for someone like me. I was also able to relate the article to past work experience when I was working AWACS missions for NORAD and had to create communication architectures on the fly daily in order to maintain a solid wide area network. The article used a broad range of examples and I was able to relate my background to several examples in the article.

rcl164's picture
Submitted by rcl164 on

The IBM article outlines the concept of architecture fairly simply early in the document, essentially stating that the system is composed of components, there are relationships among these components and there is the environment in which it all occurs.  However, the ESRI article does not explicitly describe many of the ‘parts’ and focuses on the functionality of their ArcObjects architecture.  Although the ESRI components can be inferred from the graphics in the side column, and there are a few mentions of relationships, there is a larger focus on the ‘environment’, like modularity, scalability, multi-platform support, and compatibility.

I must admit that I am not very familiar with these concepts and could only glean some basic information from these articles, but I did find two things very interesting.  First, I wonder what methods or tools the designers/developers use to incorporate the stakeholder needs?  The Eeles article points out that there can be conflicting needs or limitations based on these stakeholders, but I’m curious how these are evaluated.  Do the developers use some sort of prototyping process like we have read about in previous lessons?  Or are there some architectural industry standards that are used to guide the initial process and then tweak it in later iterations based on unique stakeholder requirements?  I just couldn’t help but think how I would incorporate all these needs and be sure that I’ve captured them all—or at least to the extent possible.  Secondly, I find the ESRI article makes it fairly clear that their system is not closed, but rather flexible, in that it allows for external development through ArcObjects.  In essence, an outside developer can add new functionality to ArcGIS by creating a new ArcObject that is attached to the current system.  This doesn’t seem anywhere near what Eeles describes.  It seems that the architecture as described in the IBM article is rigid and is designed to function as a whole without outside influence on the system once it is constructed.  I like the idea of being able to expand on a current system, like ESRI’s, or even an open-source platform, but I wonder how this influences security concerns.

hxw184's picture
Submitted by hxw184 on

You asked “Do the developers use some sort of prototyping process like we have read about in previous lessons?  Or are there some architectural industry standards that are used to guide the initial process and then tweak it in later iterations based on unique stakeholder requirements?” I would argue both are true, depending on the organization, size of the project and commitment to usability. I think there are as many methods and tools as there are designers!

I've been reading great volumes of material on methods for user-centered design (UCD) in software for the term project. From my readings, there is a long (in web terms) history of support in theory for considering stakeholder needs, and also of difficulty implementing valid, measurable UCD in practice. 

For example, back in 2002, Vredenburg, et al, did a survey of professionals in the field and came to several conclusions about the use of UCD in practice, such as  “(s)ome common characteristics of an ideal UCD process were not found to be used in practice, namely focusing on the total user experience, end-to-end user involvement in the development process, and tracking customer satisfaction”, and the issue of “the lack of measurement of UCD effectiveness and any common evaluation criteria across the industry. Respondents emphasized external objective measures but often reported the use of internal and subjective measures if any measure was used at all.”  (Vredenburg, K., Mao, J. Y., Smith, P. W., & Carey, T. (2002, April). A survey of user-centered design practice. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 471-478). ACM.)

I found this paper (and its references) through Google Scholar, and trust me there is plenty of reading on this topic up to the present day (google “agile development and user-centered design” for example). The answer to your questions as far as I can tell is yes, there are processes like we’ve read about, and yes, there are design guidelines, too.

mtd206's picture
Submitted by mtd206 on

Like several others, I have little to no exposure to the world of programming beyond the Python course I took last semester, which was interesting but did not touch on architecture much (with the exception of needing to know which ESRI library contained the objects necessary to run a program). Therefore, much of this felt quite a ways over my head as well.

However, the first thing that struck me about the Eeles article was that it started out by defining the terms used in the article, and at the end acknowledged that even within the field many of the terms are fluid and not everyone agrees upon meaning (making it even more important to define terms at the beginning of the article!). This was not at all the approach of the ESRI documentation. Precisely why ESRI is less concerned with this is unclear; perhaps it is because they are describing their system and are less concerned with how the individual elements fit into system categories.

Within the ESRI documentation, the sections did not cover just one category as described by Eeles, and in fact sometimes belie their own section titles. As an example, the very start of the documentation immediately brings hardware into the mix while discussing ArcObjects software. To some extent this is probably unavoidable, but having just read about the different categories as presented by Eeles, it seemed a bit chaotic.

The ESRI documentation covers all aspects of the systems as presented by Eeles, but again largely not in a coherent order, with the exception of the section on library organization. The focus of the documentation appears to be providing developers with an understanding of why the system architecture is the way it is and how to use it, and minimizes to some extant the what of the system.  

ezg135's picture
Submitted by ezg135 on

You made a good point.  The ESRI documentation does not follow the same coherent pattern that Eeles does.  One of the reasons, I think it might be because Eeles’ article is about describing what software architecture is, as a general concept.  The second reading focuses on describing ArcGIS architecture assuming the reader already understands the concepts that Eeles explains, or at least has some notions in this field.  For me ESRI’s document was harder to read because, not only I do not have enough knowledge about IT, but also because it describes ArcGIS architecture mentioning aspects of previous versions.  I guess if your are interested in reading ESRI’s guide is because you read the guides from previous ArcGIS releases, and that, I bet it helps to understand what we read.

Elena.

jls1082's picture
Submitted by jls1082 on

You described my impressions exactly.  The person seeking out the ESRI guide is not going to read it because they want to understand software architecture, they are reading it because they want to know specifically what changes were made in the new ArcGIS versions and how it will affect their systems.  The intended reader most likely has extensive software development experience and can decipher real meaning from the technical language throughout the paper.  Eele's article is a much more conceptual view of software design and describes what components go into it and how things relate to each other. I guess the best relation I can think of is that Eele's article is like a detailed Wikipedia page on software design, whereas ESRI is a specific case of software design.

- Jennifer 

prr123's picture
Submitted by prr123 on

This is exactly how I feel about these articles. Eele's was much more understandable to me who knows nothing about software, and the ESRI article eventually became overwhelming.

mdc305's picture
Submitted by mdc305 on

As a contrast to some others, I've written a few programs throughout my working life.  Those programs have mostly been stand-alone executables, only needed a Windows Environment to run in, and usually satisfied a particular project's needs.  With that said, coding has never been a full time job for me, though I really gave a move in that direction a lot of thought.  Because my coding wasn't done on a full-time basis, it has seemed that there was always something new to be learned.  Because of my somewhat limited experience, I’ve gained an appreciation of the bigger programming picture, especially the elements implemented by ESRI.

What I have learned is that the C++ language - used to develop ArcObjects - is a very powerful language.  It is the language that a large portion of the Windows Operating System is written in.  One of my favorite languages for coding is Visual Basic / VB.net; while it can do a lot, there are situations where C++ is the way to go.  The appeal of Visual Basic is that is an easier and faster product to begin using, while the complexity of C++ offers a deeper reach into the O/S and hardware.  A robust language like C++ was required for construction of ESRI's software architecture and certainly provided the range of functionality that was needed.

With that said, the ESRI paper details much about the ArcObjects components and how ESRI is able to use those components within several members of their product family – ArcGIS Engine, Server, and the Desktop variations.  As structural elements, those components may be used by developers outside of ESRI, without compromising the trade secrets of how the components actually function.  To me, it is strikingly similar to the internal workings of today's cars.  They are becoming more and more like black boxes: we can use them, but what makes them tick is hidden from our view.  In this case, external developers may only see the data and parameters required by an ArcGIS component in order to use it, evidence of Eeles’ behavioral elements.

Two important points from ESRI that fits into Eeles' Architecture model is that of balancing stakeholder needs and an architecture influencing its environment.  There are vendors in today's world that use the ArcGIS Engine product – it can be thought of as a gateway into that black box functionality from ESRI.  While one of the ArcGIS Desktop versions may offer too many capabilities for some customers, the ArcGIS Engine product provides a way for the vendor to tool the software to better meet their customer's needs, while still offering the core functionality from ESRI.  Thus, while only implied in their paper, ESRI is concerned with the needs of their stakeholders - specifically the vendors that provide an additional user base through the Engine product.  The topic could really have been expounded upon, in my opinion.  ESRI has provided an additional product that offers those vendors a configurable solution to, in turn, pass along to their customer base. Also, those end-use customers are helping to shape the Engine product through their use or mission of the software, as their needs are evaluated and communicated to ESRI for future software enhancements.

A point from Eeles' model that is not covered in the ESRI paper is that of behavior being defined by the architecture.  Much material is devoted to outlining the modularity of the system and how a great deal of the code has been built with reusability in mind.  However, the behavior of the software, at a user component level, isn't explored.  In Eeles' model, he provides an example with a Unified Modelling Language sequence diagram that details the interaction between different parts of a software system, at a coarse scale.  With the ESRI paper, I feel that some additional detail was needed on the behavior of finer points of the software.  How did the functionality of a set of tools develop over time?  Were they coded based upon a set of needs and never changed, or was the process iterative and went through many transitions before a final, working product was born?  Again, while the overview may have been provided, a finer view of the software architecture would have been warranted.

ezg135's picture
Submitted by ezg135 on

I really enjoy reading your post.  It really helped me to understand better the ESRI reading, I especially like the part where you make the comparison between the ArcGIS components and a black box.  It is true that cars and many appliances used to be based on simple mechanics and now they all have chips and different electronic components included, which makes it harder to repair them.  I was also told once that the car industry made most of their profits in selling parts and no so much on selling cars.  Now is you rearview mirror is broken you cannot buy just the mirror you have to buy the whole piece with the plastic part included.  I wonder is when they were designing the ArcObjects they used the same idea so they did not compromise the secret of what is “inside”.  I do not know much about how ArcGIS Engine can be customized depending on the needs of the different customers, but I think it is great that it can be done and that ESRI is getting feedback about it to improve their software.

I think the fact that you know how to write programs using C++ helps you to better understand and gives you a more complete vision of what is described by ESRI about ArcGIS architecture.

Elena.

acr181's picture
Submitted by acr181 on

I like the analogy you've used here of the black box when describing those ArcObject DLLs, Matthew. Once you get to the individual DLLs themselves, that's to me where the real and tangible difference between open and proprietary software comes into play. In the FOSS world, there aren't generally any black boxes. You can go get the source and see how something is really working. If it's not working right, you can fix it. You can extend it, too. In the proprietary structure that is used in something like ArcGIS, you might know the "hooks" into those DLLs to call them and ask them to do certain things, but you can't get under the hood to see exactly how they're working and change them from the inside.

I think you make a great point here about how showing behavior and the legacy of change are two very important elements. I think it would be really informative to know which DLLs have remained very stable over time in terms of core functionality, vs. those that are in more or less constant flux. I suppose that might reveal more than Esri would care to share though about where their work is really focused at any given time. :)

hxw184's picture
Submitted by hxw184 on

You've made me more interested in FOSS here all of sudden (fixing/improving code!). I will have to give this some more thought during this week's reading. I know I'm probably committed to Esri at work because of our time & money investment, but I would be remiss in not exploring FOSS for real, right? :-)

jln240's picture
Submitted by jln240 on

At the last session in today Dev Summit, they had an overview of what's ahead in ArcGIS.  It is very interesting to note that when it came to app development, they are really encouraging (even stressing) to use Runtime instead of on the ArcGIS Engine.  It seems like they are trying to move away from the traditional way which makes me wonder why?

frg107's picture
Submitted by frg107 on

The importance of modularity, as well as scalability, is well established in architectural designs. Similarly, software architecture will embed in the design the structural support and compromise to allow the system to be compatible, to grow and to adapt to a range of changing variables.

In my opinion, there is a limit to the architectural designers need to know details. Some level of detail might not be necessary to be accounted for. For example, to design a GIS tool using scripts language, I don’t need to worry about the little endian and big endian details. In this case, I have to focus my attention on other details of the architectural design. In the end, when I save the scripting results, I just want the data to be written to the disk, and expect to be able to extract it in a similar way. From a developer’s point of view, my preoccupation at this level should be minimal. Since we aren't defining the platform rules, we are just building on top of a platform that was built by other architectures. And for that same reason, we are limited by the platform that we choose as our base.
 
In the building industry, they can have architects who probably never build a house in their life and only handles design work, but not in the software development field. The software architect needs to understand the limitation of the platform that he or she is building. It's part of his responsibility to transfer this knowledge to the client.  For example, in traditional architecture, the clients don't ask for a change in the law of gravity after you have finished the design. Or expect the building structure to work on another planet. However, a software architecture is often asked to make such changes. Such as changing the OS or expecting that the system will work in UNIX exactly as it works in windows.
 
In the following link, Todd Merritt and Software engineer in Microsoft technologies that specializes in architectural design present a set of analogies between traditional and software architecture. Some of Todd's ideas support my comments while others, challenge my opinion. 
 
 
 

 

Pages

Subscribe to Comments for "Reading Assignment"