Exercise 3b: Transforming data and using Visualizer

Complexity: Advanced Data Requirement: ArcGIS Tutorial Data for Desktop Data Path: C:\arcgis\ArcTutor\Data Interoperability\ Goal: Learn the fundamentals of transforming features and attributes in FME Workbench and using Visualizer to verify output.

A transformer is an FME Workbench object that performs a particular restructuring of features or schema. It is a visual object that you add to your workspace canvas and connect between source readers and writer feature types or to other transformers.

This exercise introduces you to the fundamentals of using FME transformers by adding them to the spatial ETL tool you created in "Exercise 3a: Getting started with spatial ETL." You create a spatial ETL tool that transforms MGE parcel data, evaluates the new features with a test clause, and generates output as failed or passed features. Finally, you inspect the output of your workspace without generating data using Visualizer. You complete the exercise by exporting the workspace as a custom format.

Using FME Workbench, you will perform the following tasks:

• Add an AreaBuilder transformer to create polygons from the LOT_LINES source feature type.
• Add a PointOnAreaOverlayer transformer to overlay the LOT_POINTS features onto the polygon features and merge their attributes.
• Add a Tester transformer to evaluate and count the point features inside each polygon feature.
• Redirect the output of the workspace to Visualizer and inspect the output of your workspace.
• Export the workspace as a custom format.

Start ArcMap

Prerequisite:

The Data Interoperability extension must be enabled.

Before starting this exercise, you must have completed the previous exercise, Exercise 3a: Getting started with spatial ETL.

Steps:

1. Start ArcMap.
2. Click the Catalog Window button on the Standard toolbar.
3. Type C:\arcgis\ArcTutor\Data Interoperability\parcels\ in the Location text box and press ENTER.

   The location is added to the Catalog tree.

Copy and rename the spatial ETL tool

Steps:

1. Right-click the Import MGE Parcel tool and click Copy on the context menu.
2. Right-click the Parcels toolbox and click Paste on the context menu.
3. Right-click the Import MGE Parcel (2) tool, click Rename on the context menu, then type Custom Parcels.
4. Right-click the Custom Parcels tool and click Edit.

   Workbench starts.

Add an AreaBuilder transformer

The AreaBuilder transformer creates topologically correct polygon features from lines that form closed shapes.

Connect an AreaBuilder transformer to the LOT_LINES GeometryFilter.

Steps:

1. Confirm that Transformer Gallery is available.
   To open the Transformers window from the main menu, click View > Windows > Transformer Gallery.

2. In the Transformers window, type areabuilder in the Search text box and press ENTER.
3. Double-click AreaBuilder.

   An AreaBuilder transformer is added to the canvas.

4. Click the line connecting the LOT_LINES GeometryFilter and the LOT_LINES destination feature and press DELETE.

   The GeometryFilter is disconnected from the LOT_LINES writer feature type.

5. On the LOT_LINES GeometryFilter, click the yellow arrow beside the LINE attribute output and drag the arrow to the red arrow beside the AreaBuilder input.

   The GeometryFilter output is connected to the AreaBuilder input, and the arrows turn green.

Add a PointOnAreaOverlayer transformer

The PointOnAreaOverlayer transformer overlays point features onto area features and merges their attributes. As a result, each polygon feature acquires attributes from the point feature contained within its boundary. Attributes with the same name are not replaced.

Connect a PointOnAreaOverlayer transformer to the LOT_LINES AreaBuilder transformer and the LOT_POINTS GeometryFilter.

Steps:

1. In the Transformers window, type PointOnArea in the search text box and press ENTER.
2. Double-click PointOnAreaOverlayer.

   A PointOnAreaOverlayer transformer is added to the canvas.

3. On the AreaBuilder transformer, click the yellow arrow beside the AREA attribute output and drag the arrow to the red arrow beside the PointOnAreaOverlayer AREA input.

   The AreaBuilder AREA attribute output is connected to the PointOnAreaOverlayer AREA input, and the arrows turn green.

4. Click the Properties button and view the parameters.

   The PointOnAreaOverlayer Parameters dialog box opens.

   Overlap Count Attribute counts the number of point features contained in each polygon. This is useful for validating data and sending features to separate outputs for further processing. In the next section, you will connect a Tester transformer to evaluate this number and direct output accordingly.

5. Click OK.

   The PointOnAreaOverlayer Parameters dialog box closes.

6. Click the line connecting the LOT_POINTS GeometryFilter and the LOT_POINTS writer feature type and press DELETE.

   The GeometryFilter is disconnected from the writer feature type.

7. On the LOT_POINTS GeometryFilter, click the yellow arrow beside the POINT output and drag the line to the red arrow beside the PointOnAreaOverlayer POINT input.

   The GeometryFilter POINT output is connected to the PointOnAreaOverlayer POINT input, and the arrows turn green.

Organizing the canvas layout

As connectors begin to cross over each other, you can organize your workspace so that it is easier to work with.

• To organize the canvas layout, click Tools > Auto Layout and drag the objects on the canvas similar to the following image:

Tip:

An alternate method is to right-click a connector line, click Insert Vertex on link on the context menu and add one or more vertices, then drag each vertex to route the connector around other transformers.

Add a Tester transformer

The Tester transformer evaluates one or more tests on a feature and routes the feature according to the outcome of the test(s).

Connect a Tester transformer to the PointOnAreaOverlayer transformer. Add a test clause that evaluates the overlap count attribute generated by the PointOnAreaOverlayer transformer. Send polygon features containing one point feature to the PASSED output and all others to the FAILED output.

Steps:

1. In the Transformers window, type Tester in the Search text box and click Find.
2. Double-click Tester.

   A Tester transformer is added to the canvas.

3. Connect the PointOnAreaOverlayer AREA output to the Tester INPUT.

4. Click the Properties button on the Tester transformer.

   The Tester Parameters dialog box opens.

5. Double-click the first cell in the Left Value field, click the down arrow, then choose _overlaps.
6. Double-click the first cell in the Op. field, click the arrow, then choose =.
7. Click the first cell in the Right Value field and type 1.
8. Click OK.

   The Tester Parameters dialog box closes.

Reconfigure the LOT_LINES writer feature type

Rename the LOT_LINES writer feature type to problem_polys and configure it to generate the polygon features that fail the test clause you defined in the Tester transformer. Also, add a user attribute named num_points to store the _overlap values.

Steps:

1. Connect the FAILED output on the Tester transformer to INPUT on the LOT_LINES destination feature.
2. Click the Properties button on the LOT_LINES destination feature.

   The Feature Type Properties dialog box opens.

3. Click the General tab.
4. In the Feature Type Name text box, type problem_polys.
5. Click the Allowed Geometries arrow and choose geodb_polyline.
6. Click the User Attributes tab.
7. Select all attributes and press DELETE.

   The LOT_LINES attributes are not needed.

8. Click the first empty cell in the Attribute Name field and type num_points.

   The num_points attribute is used to store the _overlay attribute values generated by the Tester transformer. This value represents the quantity of overlapping points for each failed feature.

9. Click the Data Type arrow and choose integer.
10. Click OK.

    The Feature Type Properties dialog box closes.

11. Expand the Tester transformer and connect the _overlay output to num_points on the problem_polys feature type.

    The Tester transformer and feature type connections should look similar to this:

Reconfigure the LOT_POINTS writer feature type

Rename the LOT_POINTS writer feature type to parcels and configure it to generate the polygon features that pass the test clause you defined in the Tester transformer.

Steps:

1. Connect the PASSED output on the Tester transformer to INPUT on the LOT_POINTS destination feature.
2. Click the Properties button on the LOT_POINTS destination feature.

   The Feature Type Properties dialog box opens.

3. Click the General tab.
4. In the Feature Type Name text box, type parcels.
5. Click the Allowed Geometries arrow and choose geodb_polyline.
6. Click the User Attributes tab.
7. Select only the attributes with names that start with igds_ and press DELETE.

   The igds_ attributes are symbolic names created by the FME reader to simplify the translation of the IGDS source element types to FME feature types. There is no need to include them with the destination features.

8. Click OK.

   The Feature Type Properties dialog box closes.

   The Tester transformer and writer connections should look similar to this:

Verify the output of your workspace with Visualizer

Visualizer is an in-memory viewer that enables you to inspect the output of your workspace without generating data or possibly overwriting existing data.

Redirect the output of your workspace to Visualizer and verify your results.

Steps:

1. In the Navigator window, expand Tool Parameters, right-click Destination Redirect, then click Edit Parameter from the context menu.

   The Edit Parameter dialog box opens.

2. Click the Destination Redirect arrow and choose Redirect to Visualizer.
3. Click OK.

   The Edit Parameter dialog box closes.

4. On the main menu, click File > Run Translation.

   Visualizer starts.

5. Expand the datasets in Viewspace.
6. Click the Identify tool on the toolbar, select features, then explore their attribute values displayed in the Information window.
Observe that the workspace creates polygons from line elements stored in the MicroStation DGN (.par) files and acquires the point feature attributes stored in the parcels_attr.mdb database. Also observe that the Tester transformer routed features that did not pass the test clause to the problem_polys feature type.
7. Exit Visualizer.
8. On the Workbench menu, click File > Save.

Save the spatial ETL tool as a custom format

A custom format is a translation workspace that is exported from FME Workbench as a format and utilized in the FME reader gallery. It is saved to an external .fds file and can be imported to reader galleries on other desktops.

Creating a custom format adds the format to the FME reader gallery. Once created, subsequent uses of the gallery allow you to edit the format in FME Workbench or delete it from the list. Deleting a custom format removes the .fds file from your desktop.

Steps:

1. On the Workbench menu, click File > Export as Custom Format.

   The Set Custom Format Name and Description dialog box opens.

2. In the New Custom Format Name text box, type Custom_parcels.
3. In the New Custom Format Description text box, type Tutorial exercise 3b: transforming data.
4. Click OK.

   By default, a second Workbench session starts for the custom format (.fds). There is nothing to do in this workspace.

5. Click File > Exit to close the custom format workspace.
6. Click File > Exit to close the transformation workspace.

The custom format appears as a choice in the FME reader gallery the next time you open FME Workbench or create a new interoperability connection.

This completes the exercise.

Continue to the next exercise: Exercise 3c: Using source attributes to separate data.